|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectgov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean
gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
gov.nasa.pds.harvest.crawler.PDSProductCrawler
public class PDSProductCrawler
Class that extends the Cas-Crawler to crawl a directory or PDS inventory file and register products to the PDS Registry Service.
Field Summary | |
---|---|
protected boolean |
inPersistanceMode
Flag for crawler persistance. |
protected Map<File,Long> |
touchedFiles
A map of files that were touched during crawler persistance. |
Fields inherited from class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler |
---|
DIR_FILTER, FILE_FILTER, LOG |
Fields inherited from class gov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean |
---|
MIME_TYPES_HIERARCHY |
Fields inherited from interface gov.nasa.jpl.oodt.cas.filemgr.metadata.CoreMetKeys |
---|
FILE_LOCATION, FILENAME, MIME_TYPE, PRODUCT_ID, PRODUCT_NAME, PRODUCT_RECEVIED_TIME, PRODUCT_STRUCTURE, PRODUCT_TYPE |
Constructor Summary | |
---|---|
PDSProductCrawler()
Default constructor. |
|
PDSProductCrawler(Pds4MetExtractorConfig extractorConfig)
Constructor. |
Method Summary | |
---|---|
void |
addAction(gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction action)
Adds a crawler action. |
void |
addActions(List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> actions)
Adds a list of crawler actions. |
protected void |
addKnownMetadata(File product,
gov.nasa.jpl.oodt.cas.metadata.Metadata productMetadata)
Method not implemented at the moment. |
void |
crawl(File dir)
Crawls the given directory. |
List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> |
getActions()
Gets a list of crawler actions defined for the crawler. |
protected gov.nasa.jpl.oodt.cas.metadata.Metadata |
getMetadataForProduct(File product)
Extracts metadata from the given product. |
Pds4MetExtractorConfig |
getMetExtractorConfig()
Get the MetExtractor configuration object. |
RegistryIngester |
getRegistryIngester()
Gets the registry ingester. |
String |
getRegistryUrl()
Gets the registry location. |
protected boolean |
passesPreconditions(File product)
Determines whether the supplied file passes the necessary pre-conditions for the file to be registered. |
void |
setFileFilter(List<String> filters)
Sets the file filter. |
void |
setInPersistanceMode(boolean value)
|
void |
setMetExtractorConfig(Pds4MetExtractorConfig config)
|
void |
setProperties(String registryUrl,
RegistryIngester ingester,
List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> actions)
|
void |
setRegistryUrl(String url)
Sets the registry location. |
Methods inherited from class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler |
---|
clearIngestStatus, crawl, getIngestStatus, handleFile, setActionRepo |
Methods inherited from class gov.nasa.jpl.oodt.cas.crawl.config.ProductCrawlerBean |
---|
addRequiredMetadata, getActionIds, getApplicationContext, getDaemonPort, getDaemonWait, getFilemgrUrl, getGlobalMetadata, getId, getIngester, getProductPath, getRequiredMetadata, isCrawlForDirs, isNoRecur, isSkipIngest, setActionIds, setApplicationContext, setCrawlForDirs, setDaemonPort, setDaemonWait, setFilemgrUrl, setGlobalMetadata, setId, setIngester, setNoRecur, setProductPath, setRequiredMetadata, setSkipIngest |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected boolean inPersistanceMode
protected Map<File,Long> touchedFiles
Constructor Detail |
---|
public PDSProductCrawler()
public PDSProductCrawler(Pds4MetExtractorConfig extractorConfig)
extractorConfig
- A configuration class that tells the crawler
what data product types to look for and what metadata to extract.Method Detail |
---|
public Pds4MetExtractorConfig getMetExtractorConfig()
public void setMetExtractorConfig(Pds4MetExtractorConfig config)
public void setInPersistanceMode(boolean value)
public void setRegistryUrl(String url) throws MalformedURLException
url
- A url of the registry location.
MalformedURLException
- If the given url is malformed.public String getRegistryUrl()
public RegistryIngester getRegistryIngester()
public void setFileFilter(List<String> filters)
filters
- A list of file filters.protected void addKnownMetadata(File product, gov.nasa.jpl.oodt.cas.metadata.Metadata productMetadata)
addKnownMetadata
in class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
product
- The product file.productMetadata
- The metadata associated with the product.public void crawl(File dir)
crawl
in class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
dir
- The directory to crawl.public void addAction(gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction action)
action
- A crawler action.public void addActions(List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> actions)
actions
- A list of crawler actions.public List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> getActions()
public void setProperties(String registryUrl, RegistryIngester ingester, List<gov.nasa.jpl.oodt.cas.crawl.action.CrawlerAction> actions) throws MalformedURLException
MalformedURLException
protected gov.nasa.jpl.oodt.cas.metadata.Metadata getMetadataForProduct(File product)
getMetadataForProduct
in class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
product
- A PDS file.
protected boolean passesPreconditions(File product)
passesPreconditions
in class gov.nasa.jpl.oodt.cas.crawl.ProductCrawler
product
- A file.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |