|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectgov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean
gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
gov.nasa.pds.harvest.crawler.PDSProductCrawler
public class PDSProductCrawler
Class that extends the Cas-Crawler to crawl a directory or PDS inventory file and register products to the PDS Registry Service.
Field Summary |
---|
Fields inherited from class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler |
---|
DIR_FILTER, FILE_FILTER, LOG |
Fields inherited from class gov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean |
---|
MIME_TYPES_HIERARCHY |
Fields inherited from interface gov.nasa.jpl.oodt.cas.filemgr.metadata.CoreMetKeys |
---|
FILE_LOCATION, FILENAME, MIME_TYPE, PRODUCT_ID, PRODUCT_NAME, PRODUCT_RECEVIED_TIME, PRODUCT_STRUCTURE, PRODUCT_TYPE |
Constructor Summary | |
---|---|
PDSProductCrawler(PDSMetExtractorConfig extractorConfig)
Constructor. |
Method Summary | |
---|---|
void |
addAction(gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction action)
Adds a crawler action. |
void |
addActions(List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> actions)
Adds a list of crawler actions. |
protected void |
addKnownMetadata(File product,
gov.nasa.jpl.oodt.cas.metadata.Metadata productMetadata)
Method not implemented at the moment. |
void |
clearCrawlStats()
|
void |
crawl(File dir)
Crawls the given directory. |
List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> |
getActions()
Gets a list of crawler actions defined for the crawler. |
protected gov.nasa.jpl.oodt.cas.metadata.Metadata |
getMetadataForProduct(File product)
Extracts metadata from the given product. |
PDSMetExtractorConfig |
getMetExtractorConfig()
Get the MetExtractor configuration object. |
int |
getNumBadFiles()
|
int |
getNumDiscoveredProducts()
|
int |
getNumFilesSkipped()
|
RegistryIngester |
getRegistryIngester()
Gets the registry ingester. |
String |
getRegistryUrl()
Gets the registry location. |
protected boolean |
passesPreconditions(File product)
Determines whether the supplied file passes the necessary pre-conditions for the file to be registered. |
void |
setFileFilter(List<String> filters)
Sets the file filter. |
void |
setInContinuousMode(boolean value)
|
void |
setRegistryUrl(String url)
Sets the registry location. |
Methods inherited from class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler |
---|
clearIngestStatus, crawl, getIngestStatus, handleFile, setActionRepo |
Methods inherited from class gov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean |
---|
addRequiredMetadata, getActionIds, getApplicationContext, getDaemonPort, getDaemonWait, getFilemgrUrl, getGlobalMetadata, getId, getIngester, getProductPath, getRequiredMetadata, isCrawlForDirs, isNoRecur, isSkipIngest, setActionIds, setApplicationContext, setCrawlForDirs, setDaemonPort, setDaemonWait, setFilemgrUrl, setGlobalMetadata, setId, setIngester, setNoRecur, setProductPath, setRequiredMetadata, setSkipIngest |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public PDSProductCrawler(PDSMetExtractorConfig extractorConfig)
extractorConfig
- A configuration class that tells the crawler
what data product types to look for and what metadata to extract.Method Detail |
---|
public PDSMetExtractorConfig getMetExtractorConfig()
public void setInContinuousMode(boolean value)
public int getNumDiscoveredProducts()
public int getNumBadFiles()
public int getNumFilesSkipped()
public void clearCrawlStats()
public void setRegistryUrl(String url) throws MalformedURLException
url
- A url of the registry location.
MalformedURLException
public String getRegistryUrl()
public RegistryIngester getRegistryIngester()
public void setFileFilter(List<String> filters)
filters
- A list of file filters.protected void addKnownMetadata(File product, gov.nasa.jpl.oodt.cas.metadata.Metadata productMetadata)
addKnownMetadata
in class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
product
- The product file.productMetadata
- The metadata associated with the product.public void crawl(File dir)
crawl
in class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
dir
- The directory to crawl.public void addAction(gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction action)
action
- A crawler action.public void addActions(List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> actions)
actions
- A list of crawler actions.public List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> getActions()
protected gov.nasa.jpl.oodt.cas.metadata.Metadata getMetadataForProduct(File product)
getMetadataForProduct
in class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
product
- A PDS file.
protected boolean passesPreconditions(File product)
passesPreconditions
in class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
product
- A file.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |