STAC ICD for PRIP can be downloaded from this link.
Below table is available to collect the comments on this version of the document.
| Id. | Author | Section/Page number | Title | Comment | Answer |
|---|---|---|---|---|---|
| 1 | m-mohr | p. 7 | EOPF_SPECIFICATION | Missing link, there's a <tbd> instead. | Is the mentioned document already available ? |
2 | CloudFerro | p. 21 | REQ-0050 | timestamp precision should be defined to avoid differences between PRIP implementations | Proposal to keep the same precision as for CDSE (second) |
| 3 | CloudFerro | p. 25 | REQ-0085 | thumbnail size and resolution must be defined to create consistency between product types. if no thumbnail is generated by PS then this asset should be empty or removed. whatever is included in the products and does not fit the requirements should be listed as asset type "preview" or similar. | See higher level document : STAC Requirements and Recommendations for EOF Services |
| 4 | CloudFerro | p. 25 | REQ-0090 | there are various media types within ZARR, so setting mime type of all objects to "application/vnd-zarr" may cause some strange behaviour of standard client libs | |
| 5 | CloudFerro | p. 28 | REQ-0207 | it is not needed for PRIP to enable searching by any common attribute, limited subset will suffice. PRIP API is not for end-users and used only for systematic downloads | ESA asked for this level of searching capabilities |
| 6. | CloudFerro | p. 29 | REQ-0305, REQ-0410 | multipart uploads to object storage may affect the object checksum so in some cases it may be unreliable (TBC) with multipart upload single-part upload md5 != multipart-download md5 | It's up to client's implementation and should be adressed by "tailored ICD" |
| 7 | CloudFerro | p. 31 | REQ-0391 | no relation between accounts in central identity and s3 credentials make the management of access prone to errors and delays in provisioning of access. s3 credentials should be self-managed and reflect access roles (similar to CF S3 keys manager) | No known standard to include s3 credentials into IAM → out of scope of this ICD. It must be described by infrastructure provider. |
| 8 | CloudFerro | p. 31 | REQ-0400 | regarding: " PRIP client point of view, retrieving an asset needs to: List all S3 objects sharing the prefix provided by the asset link, leveraging the ListObjectsV2 S3 operation "prefix listing is time consuming and might be expensive (and will raise the maintenance cost of the PRIP) for multiple requests and will affect timeliness. compressed product recommended. | Question already discussed with ESA, and submitted to its arbitration. Action : @ESA In which way it can raise the maintenance cost ? |
| 9. | CloudFerro | STAC-PRIP-API-REQ-0210 | Access control to the STAC Catalogue is implemented at the Collection level while CDSE manages it on the product_type level. How collection and product types will be joined? | One collection corresponds to a unique product_type. Besides, PRIP is not intended for for end-users, thus access control is less critical. | |
| 10. | CloudFerro | STAC-PRIP-API-REQ-0340 | Quota Management expressed as n terra bytes per month <- what if a reprocessing campaign is published. Should every time the quota should be adjusted? | Quota adjustement should be managed at operations level and arbitrated by ESA. Not to describe in the ICD | |
| 11. | CloudFerro | lack of requirement regarding the maximum allowable number of items returned from a STAC query | We propoe to add an additional requirement on the maximum number of returned items as configurable. Action for v1.1: @Capgemini | ||
| 12. | CloudFerro | p. 22 | publication date - Timestamp of the availability of the current Item in the PRIP instance Catalogue. update date - If STAC Item properties have been updated, timestamp of the last update. Regarding the change of the modification date it is not clear if the LTA/CDSE should re-download the file. | ||
| 13 | CloudFerro | File extension should be if the zipped/concatenated product is to be exposed. The file:size and file:checksum https://github.com/stac-extensions/file | Zarr item is provided in extended form (folders & sub-folders) → neither file extension nor checksum are relevant here | ||
| 14 | CloudFerro | Section 3.4.3.4 | Downloading ZARR content | More info needed regarding "Note: It is assumed that the AWS CLI is configured with the expected S3 API endpoint and authentication credentials." | Should depend on the provider → not candidate for the ICD. rather provided in tailored ICD |
| 15 | Cloudferro | Bulk download via a single zipped file without compressions. | Downloading multiple small zarr chunk files will generate huge overload regarding number of network requests and md5/checksums checks. | Maybe we could suggest to adapt chunks size for storage usage, in order to reduce the number of objects | |
| 16 | DLR | 1.1, 2.1, 2.2, ff. | Abbreviations | It is sufficient to explain abbreviations only once (e.g., STAC, PRIP, API). On the other hand, abbreviations must be explained at the first time they are mentioned in the text (e.g. EOF, EOPF, JSON). A list of abbreviations at the beginning of the document would also be helpful. | We will add a list of abbreviations at the beginning of the document. Action for v1.1: @Capgemini |
| 17 | DLR | 2.2/page 9 ff. | Small grammar corrections | A PRIP instance manages product for a given Sentinel Mission. → A PRIP instance manages products for a given Sentinel mission. Compared to the formal PRIP ICD, current document does not currently address following PRIP functions: → Compared to the formal PRIP ICD, the current document does not address the following PRIP functions: The entire document should be thoroughly reviewed for grammar in general and uppercase and lowercase lettering more specifically. | Corrections will be provided Action for v1.1: @Capgemini |
| 18 | DLR | STAC-PRIP-ITEM-REQ-0090 STAC Asset properties, page 25 | File extension .zarr in Local Folder path | The example for the Local Folder path should include the .zarr file extension, since the <product_id> does not contain it. <product_id>.zarr/... | Corrections will be provided Action for v1.1: @Capgemini |
| 19 | DLR | STAC-PRIP-ITEM-REQ-0100 STAC Asset href, page 26 | File extension .zarr in product path | The <product_path> should include the .zarr file extension. <product_path>=<product_id>.zarr/<folder>/... | |
| 20 | DLR | STAC-PRIP-ITEM-REQ-0101 Bucket naming convention, page 26 | Example bucket names and path | The bucket name in the text does not match the bucket name of the URLs in the table. Also, the URL of the 10m Band 2 of a S02MSIL2A product would be .../measurements/reflectance/r10m/b02. This also applies to various places in the Use case examples section, page 31 ff. | The bucket name in the table is : prip-sentinel-2-l2a About Band 2, do you mean the trailing slash is to remove ? |
| 21 | DLR | 3.4.3.2/page 34 ff. | Assets and item_assets in examples | Maybe leave out the assets and item_assets from the examples since they are not defined finally for the EOPF products? | I let Jolyon arbitrate on this one Action : @ESA |
| 22 | DLR | 3.4.3.3/page 36 | Example URL | The example is referring to https://pgstac.demo.cloudferro.com/. Maybe use an anonymous URL like in the other examples. | Correction will be provided Action for v1.1: @Capgemini |
| 23 | DLR | 2.1/page 8 | On-demand processing | The PRIP API specifies not only the access to newly published Sentinel data products, it also supports the on-demand processing of (user level) data products, as defined in the PRIP ICD ESA-EOPG-EOPGC-IF-3 section 2.1. "ODPRIP" is an extension of "PRIP" and its data structures and functions (section 4 of the PRIP ICD) should also be supported in the new STAC-based ICD. | Is ODPRIP really used until now ? Action : @ESA |
| 24 | Werum | STAC-PRIP-ITEM-REQ101 Bucket naming convention 3.3.3/page 26 | Bucket naming uniqueness | Depending on the chosen cloud provider the naming of prip-<mission>-<productType> is not possible for multiple service providers. E.g. on several CloudFerro data centers and OVHcloud the bucket names need to be unique throughout the data center or across all data centers. |