SearchEngines2

The J. Conrad Dunagan Library

Subject Directories and Search Engines

What is the Difference Between a Search Engine and a Subject Directory?

Subject directories are human centered services. The people who maintain the directories decide which sites will be linked from each service. In contrast, search engines are machine centered services. Spiders grab Web pages, computer programs index the content, and a search mechanism queries the index and ranks the results.

When Should I Use a Subject Directory?

disc When you have a broad topic or idea to research
disc When you want to see a list of sites on your topic often recommended and annotated by experts
disc When you want to retrieve a list of sites relevant to your topic, rather than keywords to retrieve relevant material
disc When you want to avoid viewing low-content documents that often turn up on search services

When Should I use a Search Engine Service?

disc When you have a narrow or obscure topic or idea to research
disc When you are looking for a specific site
disc When you want to retrieve a large number of documents on your topic
disc When you want to search for particular types of documents, file types, source locations, languages, dates last modified, etc.
disc When you want to take advantage of newer retrieval technologies such as concept clustering, ranking by popularity, link ranking, and so on

Subject Directories with an Academic Focus
	Definition: A subject directory is a service that offers a collection of links to Internet resources submitted by site creators or evaluators and organized into subject categories. Directory services use selection criteria for choosing links to include, though selectivity varies among services

In late July 2006, most major British universities formed a unified network of internet subject directories search engines, internet-based academic tutorial services, and other internet-based information named intute. A number of significant subject directories including HUMBUL: Humanities Hub and Sosig: Social Sciences Information Gateway became part of this new service.

intute: intute: Science and Technology; intute: Arts and Humanities; intute: Social Sciences; intute: Health andLife Sciences
Voice of the Shuttle: Web Page for Humanities Research: The pre-eminent humanities directory.
The Best Information on the Net (BIOTN): Originally maintained by a librarian at Ambrose University, it has a section entitled "Hot Paper Topics" which focuses on current events.
BUBL: Catalogue of Selected Internet Resources: BUBL stands for Bulletin Board for Libraries and is maintained by the Andersonian Library of the University of Strathclyde in Glasgow, Scotland. This is a good site to explore for a European slant on quality Internet sites.
INFOMINE: Scholarly Internet Resource Collections: The California Digital Library. This is a project that began at UC-Riverside and has now expanded to include editors across the libraries of the University of California system.
Librarians' Index to the Internet: This is a growing high-quality subject collection, supported by a federal grant and housed at the Digital Library SunSITE at the University of California Berkeley.
PINAKES: A Subject Launchpad: Maintained at Heriot-Watt University in Edinburgh, Scotland. Provides a good entry to numerous subject gateways such as Biz/ed (business & economics), iLoveLanguages, Internet Directory for Botany, and so on.
Scout Project: This site contains links to all internet sites reviewed in The Scout Report provided by the University of Wisconsin-Madison since 1994.
WWW Virtual Library: This is the oldest academic subject directory on the World Wide Web. It is a highly respected directory sponsored by the W3 Consortium. Many of the guides are very comprehensive, however some guides have languished and are not frequently updated. Examine each subject guide with care.

Search Engine Services

Definition: A search engine service provides a searchable database of Internet files collected by a computer program, called a wanderer, crawler, robot, worm or spider. Indexing is created from the collected files, and the results are presented in a schematic order. There is no selection criteria for the collection of files.
A search service therefore consists of three components: (1) a spider, a program that traverses the Web from link to link, identifying and reading pages; (2) an index, a database containing a copy of each Web page gathered by the spider; and (3) a search engine mechanism, software that enables users to query the index and that returns results in a schematic order.

disc disc The Best Search Engines from the UC-Berkeley libraries.
disc disc SEW: Search Engine Watch a commerical site dedicated to keeping current on search engine functionality

Remember not to confuse Yahoo! the subject directory, with Yahoo!Search the search engine (see note above).

Easy Searcher (meta engine)

This is a site especially useful to the novice searcher. The site allows one to search on several popular search engines simultaneously (AltaVista, Google, HotBot, Lycos, msn, Yahoo!).

MetaCrawler (meta engine)

"crawls" Google, Yahoo!, About, LookSmart, Ask, and FindWhat

MetaSearch (meta engine)

Mamma, Seekz, SuperCrawler, MyWebSearch, Vivisimo, and Web Wombat

G o o g l e: Ranking scheme more complex than most with results that are often reasonably good for basic, limited academic research purposes. FYI: As Google became a publically traded company with shares offered on the stock exchange, they began accepting ads which stipulated that they (Google) would weight their search engines in such a way as to move certain vendors products to the "first" page of results whether the vendors web site was appropriate for the search or not. Yahoo!Search or Yahoo!Answers and a few other search engines have done this almost from the beginning of their business, but that is one reason searches using those engines are less relevant or precise than in the past. Over time Google's search engine and search results have been similarly watered down. Never limit academic research to Google only!
Yahoo!: Yahoo! is the largest subject directory on the web and is organized into a highly hierarchical subject tree. However, coverage is primarily oriented towards non-academic coverage, thus it was not included in the list of academic subject directories above. Moreover, as the oldest and most popular subject directory on the web the number of submissions asking to be linked through this site has become overwhelming. Many sites are rejected due to lack of time. Never constrain academic subject searches solely to Yahoo!

This page (including quoted materials) is based extensively on information from Choice: Current Reviews for Academic Libraries Web supplements I-VIII.

Page Updated: 26 June 2007
Page created by C. H. Shults:1 September 1999
Please send comments to: Webmaster

Visitors since 5/5/2006