All Packages  Class Hierarchy

Index of all Fields and Methods

A

action(Event, Object). Method in class bdd.search.Monitor

addElement(Object). Method in class bdd.util.FIFOQueue

Adds "object" to the queue.

addURL(URL). Method in class bdd.search.spider.Crawler

Takes "url_to_queue" and adds it to this Crawler's queue of URLs.

addURL(URL). Method in class bdd.search.spider.HTMLLinkExtractor

Adds "url" to the list of URLs.

addWord(String). Method in class bdd.search.spider.WordExtractor

Used internally to add a word to the list of words as they are found in the document.

analyze(String). Method in class bdd.search.spider.HTMLLinkExtractor

Analyzes "param", which should be the contents between a '<' and a '>', and adds any URLs that are found to the list of URLs.

C

clearErrorList(). Method in class bdd.search.Monitor

clearQueryList(). Method in class bdd.search.Monitor

clearURLList(). Method in class bdd.search.Monitor

countElements(). Method in class bdd.util.FIFOQueue

Returns the number of elements in this queue.

countOccurances(String). Method in class bdd.search.spider.WordExtractor

Returns a count of the number of times that "word" appears in the the document.

countWords(). Method in class bdd.search.spider.WordExtractor

Returns the number of words in this document.

Crawler(File, EnginePrefs). Constructor for class bdd.search.spider.Crawler

"working_dir" should be a directory that only this Crawler and a given Indexer will be accessing.

crawlerDone(Crawler). Method in class bdd.search.Monitor

Call this method when a crawler has finished crawling.

D

DBQuery(String, EnginePrefs). Constructor for class bdd.search.query.DBQuery

dumpResults(OutputStream). Method in class bdd.search.query.DBQuery

dumpToDatabase(DataOutputStream). Method in class bdd.search.spider.URLStatus

Creates a database containing just this URL.

E

EnginePrefs(). Constructor for class bdd.search.EnginePrefs

F

FIFOQueue(). Constructor for class bdd.util.FIFOQueue

FIFOQueue(int, int). Constructor for class bdd.util.FIFOQueue

"initial_cap" specifies the initial capacity of the queue and "grow_size" specifies the amount that the capacity grows by when capacity is reached.

finalize(). Method in class bdd.search.spider.URLStatus

Gets rid of the temporary file.

firstOccurance(String). Method in class bdd.search.spider.WordExtractor

Returns the index of "word".

G

getCacheFile(). Method in class bdd.search.spider.URLStatus

Returns the file that is used to cache the contents of this URL.

getEmailAddress(). Method in class bdd.search.EnginePrefs

getFooterFile(). Method in class bdd.search.EnginePrefs

getHeaderFile(). Method in class bdd.search.EnginePrefs

getLinkExtractor(). Method in class bdd.search.spider.URLStatus

Returns a LinkExtractor that can handle this URL's mime type.

getMainDir(). Method in class bdd.search.EnginePrefs

getMainIndex(). Method in class bdd.search.EnginePrefs

getMonitor(). Method in class bdd.search.EnginePrefs

getNotFoundFile(). Method in class bdd.search.EnginePrefs

getRulesFile(). Method in class bdd.search.EnginePrefs

The rules file contains rules which determine what URLs are allowed and what URLs whould be excluded.

getStartingFile(). Method in class bdd.search.EnginePrefs

getUserAgent(). Method in class bdd.search.EnginePrefs

getWordExtractor(). Method in class bdd.search.spider.URLStatus

Returns a WordExtractor that can handle this URL's mime type.

getWords(). Method in class bdd.search.spider.WordExtractor

Returns an Enumeration that returns each word in the document in no particular order.

getWorkingDir(). Method in class bdd.search.EnginePrefs

Returns the working directory for use by a crawler.

H

hasMoreElements(). Method in class bdd.util.FIFOQueue

Returns true if this queue contains more elements and false otherwise.

hasMoreElements(). Method in class bdd.search.spider.HTMLLinkExtractor

hasMoreElements(). Method in class bdd.search.spider.NullLinkExtractor

HTMLLinkExtractor(File, URL). Constructor for class bdd.search.spider.HTMLLinkExtractor

Creates a new HTMLLinkExtractor that will enumerate all the URLs in the give "cache_file".

HTMLWordExtractor(File). Constructor for class bdd.search.spider.HTMLWordExtractor

I

Indexer(File, Crawler, EnginePrefs). Constructor for class bdd.search.spider.Indexer

"working_dir" should be a directory that only this Indexer and a given Cralwer will be accessing.

indexing(URL). Method in class bdd.search.Monitor

Call this method when a new URL is being indexed.

L

loaded(). Method in class bdd.search.spider.URLStatus

Returns true if and only if this URL was loaded without an error.

M

main(File, EnginePrefs). Static method in class bdd.search.spider.Crawler

main(File, EnginePrefs, boolean). Static method in class bdd.search.spider.Crawler

main(String[]). Static method in class bdd.search.spider.Crawler

This is the method that is called when this class is invoked from the command line.

main(String[]). Static method in class bdd.search.query.DBQuery

This is the method that is called when this class is invoked from the command line.

main(String[]). Static method in class bdd.search.Monitor

main(String[]). Static method in class bdd.search.query.QueryWebServer

This is the method that is called when the server is started from the command line.

mimeTypeUnderstood(String). Method in class bdd.search.spider.URLStatus

Returns true if and only if this mime type can be processed.

Monitor(). Constructor for class bdd.search.Monitor

Starts a monitor with the custom web server running on the default port.

Monitor(int). Constructor for class bdd.search.Monitor

Starts a monitor with the custom web server running on the specified port.

moved(). Method in class bdd.search.spider.URLStatus

Returns true if and only if this URL causes a redirection.

N

nextElement(). Method in class bdd.util.FIFOQueue

Removes the next element from the queue and returns it.

nextElement(). Method in class bdd.search.spider.HTMLLinkExtractor

nextElement(). Method in class bdd.search.spider.NullLinkExtractor

NullLinkExtractor(). Constructor for class bdd.search.spider.NullLinkExtractor

P

pause_time. Variable in class bdd.search.EnginePrefs

The time to pause between URL fetches (in seconds).

pauseBetweenURLs(). Method in class bdd.search.EnginePrefs

Pauses for the amount of time that has been specified for pausing between URL fetches.

port. Static variable in class bdd.search.EnginePrefs

Q

QueryConnection(Socket, EnginePrefs). Constructor for class bdd.search.query.QueryConnection

querying(String). Method in class bdd.search.Monitor

Call this method when a new query is being placed.

QueryWebServer(). Constructor for class bdd.search.query.QueryWebServer

Starts a new QueryWebServer on the default port.

QueryWebServer(int). Constructor for class bdd.search.query.QueryWebServer

Starts a new QueryWebServer on the specified port.

QueryWebServer(int, EnginePrefs). Constructor for class bdd.search.query.QueryWebServer

Starts a new QueryWebServer on the specified port with the given preferences.

queueURL(URLStatus). Method in class bdd.search.spider.Indexer

Use this method to add a cached url to the Indexer.

R

readContent(). Method in class bdd.search.spider.URLStatus

Downloads the content of the given URL and stores it in a temporary cache file.

readNextElement(). Method in class bdd.util.FIFOQueue

Returns the next element that on the queue, but leaves the element on the queue so that the next call to nextElement() or readNextElement() will return the same element.

readRobotsDotText(String, int). Method in class bdd.search.EnginePrefs

Reads the "robots.txt" file on the given host and uses the results to determine what files on "host" are crawlable.

readRulesFile(). Method in class bdd.search.EnginePrefs

Causes the inclusion/exclusion rules to be read.

reportError(URL). Method in class bdd.search.Monitor

Call this method to report an error during crawling.

reset(). Method in class bdd.search.spider.HTMLLinkExtractor

Resets this enumeration.

run(). Method in class bdd.search.spider.Crawler

This is where the actual crawling occurs.

run(). Method in class bdd.search.spider.Indexer

This is where the actual indexing is done.

run(). Method in class bdd.search.query.QueryConnection

Read a query and respond appropriately.

run(). Method in class bdd.search.query.QueryWebServer

This is where connections are accepted.

S

setCapacity(int). Method in class bdd.util.FIFOQueue

Conserves memory by shrinking the capacity of the queue to "new_capacity".

shrinkCapacityToSize(). Method in class bdd.util.FIFOQueue

Sets the capacity to the number of elements in the queue.

start(). Method in class bdd.search.spider.Indexer

Starts the Indexer.

stopWhenDone(boolean). Method in class bdd.search.spider.Indexer

Causes this Indexer to stop whenever it finishes indexing the URLs in its queue.

T

TextWordExtractor(File). Constructor for class bdd.search.spider.TextWordExtractor

U

URLAllowed(URL). Method in class bdd.search.EnginePrefs

Returns true if "url" is allowed to be indexed and false otherwise.

URLStatus(URL, File, EnginePrefs). Constructor for class bdd.search.spider.URLStatus

"url" is the location of the information and "temp_file" is the temporary file that can be used to store the contents of this url.

W

WordExtractor(). Constructor for class bdd.search.spider.WordExtractor