Installation

This document describes how to install the Search Service software contained in the search-service package. The following topics can be found in this section:

System Requirements

This section details the system requirements for installing and operating the Search Service.

Java Runtime Environment

The Search Service was developed using Java and will run on any platform with a supported Java Runtime Environment (JRE). The software was specifically compiled for and tested in Java version 1.8. The following commands test the local Java installation in a UNIX-based environment:

% which java
/usr/bin/java

% java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
        

The first command above checks whether the java executable is in the environment's path and the second command reports the version. If Java is not installed or the version is not at least 1.8, Java will need to be downloaded and installed in the current environment. Consult the local system administrator for installation of this software. For the do-it-yourself crowd, the Java software can be downloaded from the Oracle Java Download page. The suggested software package is the Java Standard Edition (SE) 8, either the JDK or the JRE package. The JDK package is not necessary to run the software but could be useful if development and compilation of Java software will also occur in the current environment.

Solr Server

The Search Service requires a Solr server. The suggested Solr server for this release is a minimal version of 7.5.0. Consult the local system administrator for installation of this software. For the do-it-yourself crowd, see the Solr tutorial document for installation and configuration detail.

The top-level directory where Solr is installed (i.e. the directory containing the server and dist sub-directories) will be referenced in these instructions as $SOLR_HOME.

Unpacking the Package

Download the search-service package from the PDS FTP site. The binary distribution is available in identical zip or tar/gzip packages. The installation directory may vary from environment to environment but in UNIX-based environments it is typical to install software packages in the /usr/local directory and in Windows-based environments it is typical to install software packages in the C:\Program Files directory. Unpack the selected binary distribution file with one of the following commands:

% unzip search-service-2.0.0-bin.zip
or
% tar -xzvf search-service-2.0.0-bin.tar.gz
      

Note: Depending on the platform, the native version of tar may produce an error when attempting to unpack the distribution file because many of the file paths are greater than 100 characters. If available, the GNU version of tar will resolve this problem. If that is not available or cannot be installed, the zipped package will work just fine in a UNIX environment.

The commands above result in the creation of the search-service-2.0.0 directory with the following directory structure:

  • LICENSE.txt

    The copyright notice from the California Institute of Technology detailing the restrictions regarding the use and distribution of this software. Although the license is strictly worded, the software has been classified as Technology and Software Publicly Available (TSPA) and is available for anyone to download and use.

  • README.txt

    A README file directing the user to the available documentation for the project.

  • dist/

    This directory contains the jar for the Search Service API (PDS Search Protocol).

  • conf/

    This directory contains the configuration for the Solr core for the PDS Search. This directory path is the SEARCH_HOME referenced in the Search Core Operation documentation

  • lib/

    This directory contains the jars used by the Solr to parse XSLT.

Install Solr and Deploy the PDS Configuration

The Search Service includes the configuration for Solr as well as the Java extensions written to support the PDS Search Protocol and PDAP Handler. In order to deploy Solr with the Search Service configuration, please complete the following procedure:

Solr Installation and Configuration

  1. Create a symbolic link to the Search Service installation directory to provide an easier method of upgrade in the future. The following demonstrates how to create this symbolic link:
    ln -s /usr/local/search-service-2.0.0 /usr/local/search-service
                
  2. Install Solr as indicated in Installing Solr document.

    Unpack SOLR pkg

    tar xvf solr-7.6.0.tgz
    cd solr-7.6.0
                
  3. Copy saxon*.jar to solr-webapp directory
  4. cp /usr/local/search-service/lib/saxon*.jar $SOLR_HOME/server/solr-webapp/webapp/WEB-INF/lib/
              
  5. Copy the Search API JAR to necessary location for Solr to recogize it:
  6. mkdir -p $SOLR_HOME/contrib/pds/lib
    cp /usr/local/search-service/dist/*.jar $SOLR_HOME/contrib/pds/lib
              
  7. Copy 'pds' config set to necessary location for Solr to recognize it:
  8. mkdir -p $SOLR_HOME/server/solr/configsets/pds
    cp -pr /usr/local/search-service/conf $SOLR_HOME/server/solr/configsets/pds/conf
                
  9. Start the Solr server with 'javax.xml.transform.TransformerFactory' and 'solr.pds.home' properties in CLOUD mode
  10. ./bin/solr start -c -a "-Djavax.xml.transform.TransformerFactory=net.sf.saxon.TransformerFactoryImpl \
    -Dsolr.pds.home=$SOLR_HOME/server/solr/ -Xmx2048m" -s $SOLR_HOME/server/solr
                
  11. Create a PDS collection
  12. ./bin/solr create -c pds -d pds
                
  13. Create a Registry collection
  14. First create the .system collection, which is required by Solr to be able to store the product labels.

    curl "http://localhost:8983/solr/admin/collections?\
    action=CREATE&name=.system&replicationFactor=1&numShards=1&maxShardsPerNode=1"
                

    You should see something similar to the following:

    {
      "responseHeader":{
        "status":0,
        "QTime":2603},
      "success":{
        "137.79.199.178:8983_solr":{
          "responseHeader":{
            "status":0,
            "QTime":1217},
          "core":".system_shard1_replica_n1"}},
      "warning":"Using _default configset. Data driven schema functionality is enabled by default,\
       which is NOT RECOMMENDED for production use. To turn it off: \
       curl http://{host:port}/solr/.system/config -d '{\"set-user-property\": {\"update.autoCreateFields\":\"false\"}}'"}
                

    Next, create the registry alias.

    curl "http://localhost:8983/solr/admin/collections?\
    action=CREATEALIAS&name=registry&collections=.system"
                

    You should see something similar to the following:

    {
      "responseHeader":{
        "status":0,
        "QTime":112}}
                
  15. Test installation by pointing your browser to http://localhost:8983/solr/ (Domain and port will vary according to Solr configuration)

Install and Deploy Solr in Docker

This section details how to install and deploy a dockerized instance of the PDS Search Service. It has been separated into 2 sections depending on the target platform.

Windows Users

Note that this installation is still in its experimental stages and has not been fully tested.

  1. Install Curl if needed.

    If Curl needs to be installed on the target Windows machine, it can be downloaded from the Curl website. Make sure that the PATH environment references the bin/ folder in the Curl directory(i.e. C:\path\to\curl\bin).

  2. Install Docker Toolbox.

    Install Docker Toolbox from the Docker website.

  3. Configure the VM.

    Oracle VM VirtualBox should have been installed from step 1). Open up this application by clicking on the Oracle VM VirtualBox Manager icon. The VM settings configured for Docker should be optimized to boost the performance of the Search Service. To do this, right-click on the VM that is labeled default, then click on Settings.

    Once inside the Settings window, click on System and in the Motherboard tab, increase the Base Memory accordingly:

    Next, click on the Processor tab and increase the number of Processors to use for the VM:

    Click on the Network menu icon, click the arrow next to the Advanced section, and click on the Port Forwarding button.

    Add a new Port Forwarding rule, by clicking on the + icon on the right. Set the Name field to Search Service, and set the Host Port and Guest Port fields to 8983.

  4. Once the VM is configured, click on the Docker Quickstart Terminal Windows icon.

  5. Run the deploy script.

    The Docker Terminal runs in a Linux-like environment. The VM mounts the user's home directory and it can be accessed in a Linux-style path. So C:\Users\PDS_User can be accessed through this Terminal Window like so: /C/Users/PDS_User.

    It is recommended that the Search Service package be installed under the user's home directory. Assuming that the Search Service package is installed under C:\Users\PDS_User\search-service, do the following:

    $ cd /C/Users/PDS_User/search-service/build
    
    $ ./deploy-docker-win.sh install
              

    You should see output similar to the following:

    Building Registry/Search Docker Image.      SUCCESS
    Starting Registry/Search Docker Container.  SUCCESS
    Waiting for Solr server to start...
    Creating the Registry collection.         SUCCESS
    Creating the Search collection.           SUCCESS
              
  6. Verify installation.

    Open up your favorite browser and type in http://localhost:8983/solr. You should see the Solr admin interface.

When the deploy script is run, it will output a log file. In this log file, it is expected to see the following message:

SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. \
All files and directories added to build context will have '-rwxr-xr-x' permissions. \
It is recommended to double check and reset permissions for sensitive files and directories.        
        

At the time of this writing, it is unknown what the cause of this issue is. It does not appear to affect the Search Service in the testing that was performed.

Common Errors

HTTP 404 - Requested Resource Not Available

This error means the application failed to install correctly. The following are potential mitigation strategies:

  • Verify domain and port are correct for the local Solr installation.
  • View $SOLR_HOME/server/logs/solr.log for information on potential causes of the failed deployment.

Other Errors

For other errors that arise while testing the installation, see the Operate - Common Errors