2023.05.13 curl 'https://pds.nasa.gov/services/search/search?wt=json&q=search_id:pds3:urn:nasa:pds:context_pds3:resource:resource.con-cal-cfi-1-edr-ground-ocf-v1.0__browserp_con-cal-cfi-1-edr-ground-ocf-v1.0' curl 'https://pds.nasa.gov/services/search/search?wt=json&q=search_id:pds3:urn:nasa:pds:context_pds3:resource:resource.con-cal-tlm-1-edr-ground-gse-v1.0__browserp_con-cal-tlm-1-edr-ground-gse-v1.0' both resulted in search results with file_ref_url with starbase. So I will reingest CON-CAL-CFI-1-EDR-GROUND-OCF-V1.0 CON-CAL-TLM-1-EDR-GROUND-GSE-V1.0 because no one's going to miss them if I muck this up. I wanted the other two non-doc CONTOUR datasets, but 1 or both had the wrong DATA_SET_ID or something egregious. I deleted the first dataset from the resource manager as well. If both curls end up with no starbase, I won't delete from the resource manager later, when I try to erase all traces of starbase from the new db. From: "Padams, Jordan H (US 398A)" To: "Chen, Richard L (US 398N)" Subject: RE: Search Core Config Date: Thu, 11 May 2023 14:02:44 +0000 Copy that. Thanks Richard. I agree PDS3 ingestion is a bit of a black box right now, so I think we can reprioritize that for the time being. The PDS4 stuff may be easier to update in the near term since search core reloads all the data every time. Jordan -- Jordan Padams Engineering Node Manager Planetary Data System (PDS) NASA Jet Propulsion Laboratory jordan.h.padams@jpl.nasa.gov 626.390.9480 -------- Original message -------- From: "Chen, Richard L (US 398N)" Date: 5/11/23 2:59 AM (GMT-05:00) To: "Padams, Jordan H (US 398A)" Subject: Re: Search Core Config There's a lot of magic for PDS3 files that I don't know about. Here's my current guess that I'll test out: "starbase" mainly (only?) appears in the file_ref_url for the resource products for the datasets that migrated from the PDS3 db. % curl 'https://pds.nasa.gov/services/search/search?wt=json&q=search_id:pds3:urn:nasa:pds:context_pds3:resource:resource.nh-a-alice-3-kem1-v6.0__browserp_nh-a-alice-3-kem1-v6.0' which is for a dataset ingested for the first and only time in Feb 2023. It has no fields named file_anything, and no fields with "starbase" as part of its value. Searching for "New Horizons Kuiper Extended Mission ALICE" returns something that points to the right place at SBN. So I'm going to reingest 2 old PDS3 datasets. For 1 of them, I'll also delete its existence from the Resource Manager first in case the magic goes there. I'll do this on pdscloud-prod1 and see what the curl returns for both in the end. Too tired to do that clear-headedly now. Richard On 5/9/23, 15:35, "Padams, Jordan H (US 398A)" wrote: Hey Richard, I am actually seeing a lot of these with Bundles and Collections and Context products? I understand PDS3 data sets may be quirky, so we can leave those for now. Jordan -- Jordan Padams Engineering Node Manager Planetary Data System (PDS) NASA Jet Propulsion Laboratory jordan.h.padams@jpl.nasa.gov > 626.390.9480 From: Richard Chen > Date: Tuesday, May 9, 2023 at 2:16 PM To: "Padams, Jordan H (US 398A)" > Subject: Re: Search Core Config Sorry I've been dragging my feet on this. It's a huge task because it requires reingesting all catalog files for the affected datasets, which typically have syntactic errors and warnings that must get investigated/resolved. This has been on my back burner for a while. I'll try to bump it up. Richard On 5/9/23, 12:29, "Padams, Jordan H (US 398A)" > wrote: Hey Richard, It looks like our search core or harvest config is still referencing starbase. Can we update those? For instance, search for file_ref_url here: https://pds.nasa.gov/services/search/search?wt=json&q=product_class:Product_Collection%20or%20product_class:Product_Bundle%20or%20product_class:Product_Data_Set_PDS3 . Any thoughts?e Thanks, Jordan -- Jordan Padams Engineering Node Manager Planetary Data System (PDS) NASA Jet Propulsion Laboratory jordan.h.padams@jpl.nasa.gov 626.390.9480