IFS

Ingesting IFS data products into Data Central allows for population of the Single Object Viewer, from which users are able to download and explore (file type dependent) those data products. The Schema Browser will also include an entry for each IFS data product type. The minimum requirements for ingesting IFS data products are described below.

Note

Please be aware that this is a living document. Although Data Central will take every effort to minimise the changes to the format of the metadata required, we may from time to time make changes to these requirements, in particular the data product visualisation requirements for 2D maps, in order to deliver the best possible user experience.

Note

Remember that the documentation mentioned here is uneditable once the data are released. Documentation intended for public use and/or subject to change (e.g., with detailed descriptions of analysis or version update information) should be maintained by the survey teams in Data Central’s Documentation portal: Documentation Central.

Data Model

Data Central’s ingestion process will map your data onto the Data Central data model. Within Data Central, IFS data are organised hierarchically, as per:

<survey>
└── <datarelease>
    └── Schema:IFS
        └── Facility
            └── Data Product

To explore the data model further, visit the IFS section of the Schema Browser to explore the relationships between facilities and data products.

Directory Structure

To ingest IFS data products, you will provide two folders, one containing the data products themselves, and one containing the metadata.

Data

The ifs directory should contain the FITS files themselves.

data
└── <survey>
    └── <datarelease>
        └── ifs
            └── <source_name>
                ├── product1.fits
                └── product2.gz
            └── <source_name>
                ├── product3.fits
                └── product4.fits

Attention

<survey> and <datarelease> should be replaced with the values you chose in Getting Started, e.g., gama and dr2.

A good rule of thumb is to keep your files succinct and with as few extensions as possible (i.e, separate your extensions into different files). This makes mapping the correct file (by data type) to a browser visualiser simpler. Currently we support 2D map visualisations in the browser. A good example is a SAMI DR2 object, which you can see in the Single Object Viewer.

Metadata

The following file structure should be adopted. A top-level <survey> directory should contain a single directory per <datarelease>. Please follow the directory format indicated below. The metadata files will populate the Schema Browser.

dcmetadata
└── surveys
    |____ <survey>
    ├──── <survey>_survey_meta.txt
    └──── <datarelease>
          ├── <survey>_<datarelease>_data_release_meta.txt
          └── ifs/
              ├── <survey>_<datarelease>_facility_meta.txt
              ├── <survey>_<datarelease>_group_meta.txt
              ├── <survey>_<datarelease>_product_ingestion.txt
              ├── <survey>_<datarelease>_product_meta.txt
              └── docs/                                         ** optional

The dcmetadata/surveys/<survey>/<datarelease>/ifs/ directory will contain a minimum of 4 metadata files, plus an optional docs/ directory if you have supplied additional documentation for a particular facility/product.

Metadata Files

Attention

Metadata files are always pipe-delimited, and have the extension .txt.

<survey>_<datarelease>_facility_meta.txt

This file describes the facilities you would like to register, and will be used to populate the Schema Browser and SOV.

Please provide a single pipe-delimited .txt file containing an entry (row) for each facility, in the following format:

name	pretty_name	description	documentation
sami	SAMI	Brief SAMI instrument description	sami_instrument_doc.txt

Please name this file: <survey>_<datarelease>_facility_meta.txt, replacing <survey> and <datarelease> with the survey name and datarelease name, respectively: e.g., sami_dr2_facility_meta.txt.

<survey>_<datarelease>_facility_meta.txt

This file should contain the following columns:

name(required=True, type=string, max_limit=100): Facility name. Use only alphanumeric characters. This must be unique per data release.

pretty_name(required=True, type=string, max_limit=500): A human-readable version of the facility name. This can contain any characters (up to the character limit).

description(required=True, type=string, max_limit=500): A succinct paragraph describing the facility.

documentation(required=False, type=string, max_limit=1000): If you would like formatted text to appear in the Schema Browser, you must supply a file in the docs/ directory. The file can be either plain text or markdown-formatted text (see Formatting for more info). This field is set to the name of the file. If you do not wish to supply documentation for a particular group, leave this entry blank. Remember that the documentation mentioned here is uneditable once the data are released. Documentation intended for public use and/or subject to change (e.g., with detailed descriptions of analysis or version update information) should be maintained by the survey teams in Data Central’s Documentation portal: Documentation Central.

<survey>_<datarelease>_product_meta.txt

This file describes the types of data products you would like to register, and will be used to populate the Schema Browser and SOV.

Please provide a single pipe-delimited .txt file containing an entry (row) for each product, in the following format:

facility_name	name	description	documentation	group_name	version	contact
sami	spectral_cube-blue	Fully reduced and flux calibrated SAMI cube (blue)	spec_cube_blue.md	cube	1.2	John Smith <john.smith@institute.org>
sami	sfr_default_1-comp	1-component star formation rate map from default cube	sfr.md	sfr	1.2	John Smith <john.smith@institute.org>

Please name this file: <survey>_<datarelease>_product_meta.txt, replacing <survey> and <datarelease> with the survey name and datarelease name, respectively: e.g., sami_dr2_product_meta.txt.

<survey>_<datarelease>_product_meta.txt

This file should contain the following columns:

facility_name(required=True, type=string, max_limit=100): The name of the facility (this must match a facility name from the <survey>_<datarelease>_facility_meta.txt file above).

name(required=True, type=string, max_limit=100)

Data product name. Choose from the following names:

name	pretty_name	vis_type
cube_blue	Flux Cube: Blue	3d_cube
adaptive_blue	Adaptively-binned Flux Cube: Blue	3d_cube
annular_blue	Annularly-binned Flux Cube: Blue	3d_cube
sectors_blue	Sectors-binned Flux Cube: Blue	3d_cube
cube_red	Flux Cube: Red	3d_cube
adaptive_red	Adaptively-binned Flux Cube: Red	3d_cube
annular_red	Annularly-binned Flux Cube: Red	3d_cube
sectors_red	Sectors-binned Flux Cube: Red	3d_cube
spectrum_3-kpc_blue	3kpc Aperture Spectrum: Blue	1d_spectrum
spectrum_re_blue	Re Elliptical Aperture Spectrum: Blue	1d_spectrum
spectrum_1-4-arcsec_blue	1”.4 Aperture Spectrum: Blue	1d_spectrum
spectrum_2-arcsec_blue	2” Aperture Spectrum: Blue	1d_spectrum
spectrum_3-arcsec_blue	3” Aperture Spectrum: Blue	1d_spectrum
spectrum_4-arcsec_blue	4” Aperture Spectrum: Blue	1d_spectrum
spectrum_3-kpc_red	3kpc Aperture Spectrum: Red	1d_spectrum
spectrum_re_red	Re Elliptical Aperture Spectrum: Red	1d_spectrum
spectrum_1-4-arcsec_red	1”.4 Aperture Spectrum: Red	1d_spectrum
spectrum_2-arcsec_red	2” Aperture Spectrum: Red	1d_spectrum
spectrum_3-arcsec_red	3” Aperture Spectrum: Red	1d_spectrum
spectrum_4-arcsec_red	4” Aperture Spectrum: Red	1d_spectrum
stellar-velocity_default_two-moment	Stellar Velocity Map	2d_map
stellar-velocity-dispersion_default_two-moment	Stellar Velocity Dispersion Map	2d_map
stellar-velocity_adaptive_two-moment	Stellar Velocity Map from Adaptively-binned Cube	2d_map
stellar-velocity-dispersion_adaptive_two-moment	Stellar Velocity Dispersion Map from Adaptively-binned Cube	2d_map
stellar-velocity_annular_two-moment	Stellar Velocity Map from Annularly-binned Cube	2d_map
stellar-velocity-dispersion_annular_two-moment	Stellar Velocity Dispersion Map from Annularly-binned Cube	2d_map
stellar-velocity_sectors_two-moment	Stellar Velocity Map from Sectors-binned Cube	2d_map
stellar-velocity-dispersion_sectors_two-moment	Stellar Velocity Dispersion Map from Sectors-binned Cube	2d_map
gas-velocity_default_1-comp	1-component Ionised Gas Velocity Map	2d_map
gas-vdisp_default_1-comp	1-component Ionised Gas Velocity Dispersion Map	2d_map
OII3728_default_1-comp	1-component Line Emission Map: [OII] (3726Å+3729Å)	2d_map
Halpha_default_1-comp	1-component Line Emission Map: Hα	2d_map
Hbeta_default_1-comp	1-component Line Emission Map: Hβ	2d_map
OIII5007_default_1-comp	1-component Line Emission Map: [OIII] (5007Å)	2d_map
OI6300_default_1-comp	1-component Line Emission Map: [OI] (6300Å)	2d_map
NII6583_default_1-comp	1-component Line Emission Map: [NII] (6583Å)	2d_map
SII6716_default_1-comp	1-component Line Emission Map: [SII] (6716Å)	2d_map
SII6731_default_1-comp	1-component Line Emission Map: [SII] (6731Å)	2d_map
gas-velocity_default_recom-comp	Recommended-component Ionised Gas Velocity Map	2d_map
gas-vdisp_default_recom-comp	Recommended-component Ionised Gas Velocity Dispersion Map	2d_map
OII3728_default_recom-comp	Recommended-component Line Emission Map: [OII] (3726Å+3729Å)	2d_map
Halpha_default_recom-comp	Recommended-component Line Emission Map: Hα	2d_map
Hbeta_default_recom-comp	Recommended-component Line Emission Map: Hβ	2d_map
OIII5007_default_recom-comp	Recommended-component Line Emission Map: [OIII] (5007Å)	2d_map
OI6300_default_recom-comp	Recommended-component Line Emission Map: [OI] (6300Å)	2d_map
NII6583_default_recom-comp	Recommended-component Line Emission Map: [NII] (6583Å)	2d_map
SII6716_default_recom-comp	Recommended-component Line Emission Map: [SII] (6716Å)	2d_map
SII6731_default_recom-comp	Recommended-component Line Emission Map: [SII] (6731Å)	2d_map
gas-velocity_adaptive_1-comp	1-component Ionised Gas Velocity Map from Adaptively-binned Cube	2d_map
gas-vdisp_adaptive_1-comp	1-component Ionised Gas Velocity Dispersion Map from Adaptively-binned Cube	2d_map
OII3728_adaptive_1-comp	1-component Line Emission Map from Adaptively-binned Cube: [OII] (3726Å+3729Å)	2d_map
Halpha_adaptive_1-comp	1-component Line Emission Map from Adaptively-binned Cube: Hα	2d_map
Hbeta_adaptive_1-comp	1-component Line Emission Map from Adaptively-binned Cube: Hβ	2d_map
OIII5007_adaptive_1-comp	1-component Line Emission Map from Adaptively-binned Cube: [OIII] (5007Å)	2d_map
OI6300_adaptive_1-comp	1-component Line Emission Map from Adaptively-binned Cube: [OI] (6300Å)	2d_map
NII6583_adaptive_1-comp	1-component Line Emission Map from Adaptively-binned Cube: [NII] (6583Å)	2d_map
SII6716_adaptive_1-comp	1-component Line Emission Map from Adaptively-binned Cube: [SII] (6716Å)	2d_map
SII6731_adaptive_1-comp	1-component Line Emission Map from Adaptively-binned Cube: [SII] (6731Å)	2d_map
gas-velocity_annular_2-comp	2-component Ionised Gas Velocity Map from Annularly-binned Cube	2d_map
gas-vdisp_annular_2-comp	2-component Ionised Gas Velocity Dispersion Map from Annularly-binned Cube	2d_map
OII3728_annular_2-comp	2-component Line Emission Map from Annularly-binned Cube: [OII] (3726Å+3729Å)	2d_map
Halpha_annular_2-comp	2-component Line Emission Map from Annularly-binned Cube: Hα	2d_map
Hbeta_annular_2-comp	2-component Line Emission Map from Annularly-binned Cube: Hβ	2d_map
OIII5007_annular_2-comp	2-component Line Emission Map from Annularly-binned Cube: [OIII] (5007Å)	2d_map
OI6300_annular_2-comp	2-component Line Emission Map from Annularly-binned Cube: [OI] (6300Å)	2d_map
NII6583_annular_2-comp	2-component Line Emission Map from Annularly-binned Cube: [NII] (6583Å)	2d_map
SII6716_annular_2-comp	2-component Line Emission Map from Annularly-binned Cube: [SII] (6716Å)	2d_map
SII6731_annular_2-comp	2-component Line Emission Map from Annularly-binned Cube: [SII] (6731Å)	2d_map
gas-velocity_sectors_1-comp	1-component Ionised Gas Velocity Map from Sectors-binned Cube	2d_map
gas-vdisp_sectors_1-comp	1-component Ionised Gas Velocity Dispersion Map from Sectors-binned Cube	2d_map
OII3728_sectors_1-comp	1-component Line Emission Map from Sectors-binned Cube: [OII] (3726Å+3729Å)	2d_map
Halpha_sectors_1-comp	1-component Line Emission Map from Sectors-binned Cube: Hα	2d_map
Hbeta_sectors_1-comp	1-component Line Emission Map from Sectors-binned Cube: Hβ	2d_map
OIII5007_sectors_1-comp	1-component Line Emission Map from Sectors-binned Cube: [OIII] (5007Å)	2d_map
OI6300_sectors_1-comp	1-component Line Emission Map from Sectors-binned Cube: [OI] (6300Å)	2d_map
NII6583_sectors_1-comp	1-component Line Emission Map from Sectors-binned Cube: [NII] (6583Å)	2d_map
SII6716_sectors_1-comp	1-component Line Emission Map from Sectors-binned Cube: [SII] (6716Å)	2d_map
SII6731_sectors_1-comp	1-component Line Emission Map from Sectors-binned Cube: [SII] (6731Å)	2d_map
gas-velocity_adaptive_recom-comp	Recommended-component Ionised Gas Velocity Map from Adaptively-binned Cube	2d_map
gas-vdisp_adaptive_recom-comp	Recommended-component Ionised Gas Velocity Dispersion Map from Adaptively-binned Cube	2d_map
OII3728_adaptive_recom-comp	Recommended-component Line Emission Map from Adaptively-binned Cube: [OII] (3726Å+3729Å)	2d_map
Halpha_adaptive_recom-comp	Recommended-component Line Emission Map from Adaptively-binned Cube: Hα	2d_map
Hbeta_adaptive_recom-comp	Recommended-component Line Emission Map from Adaptively-binned Cube: Hβ	2d_map
OIII5007_adaptive_recom-comp	Recommended-component Line Emission Map from Adaptively-binned Cube: [OIII] (5007Å)	2d_map
OI6300_adaptive_recom-comp	Recommended-component Line Emission Map from Adaptively-binned Cube: [OI] (6300Å)	2d_map
NII6583_adaptive_recom-comp	Recommended-component Line Emission Map from Adaptively-binned Cube: [NII] (6583Å)	2d_map
SII6716_adaptive_recom-comp	Recommended-component Line Emission Map from Adaptively-binned Cube: [SII] (6716Å)	2d_map
SII6731_adaptive_recom-comp	Recommended-component Line Emission Map from Adaptively-binned Cube: [SII] (6731Å)	2d_map
extinct-corr_default_1-comp	Extinction correction map from 1-component Hα/Hβ flux ratio	2d_map
sfr-mask_default_1-comp	Star formation mask map from 1-component emission line ratios	2d_map
sfr_default_1-comp	Star formation rate map from 1-component Hα flux	2d_map
sfr-dens_default_1-comp	Star formation rate surface density from 1-component Hα flux	2d_map
extinct-corr_default_recom-comp	Extinction correction map from recommended-component Hα/Hβ flux ratio	2d_map
sfr-mask_default_recom-comp	Star formation mask map from recommended-component emission line ratios	2d_map
sfr_default_recom-comp	Star formation rate map from recommended-component Hα flux	2d_map
sfr-dens_default_recom-comp	Star formation rate surface density from recommended-component Hα flux	2d_map
gas-velocity_sectors_recom-comp	Recommended-component Ionised Gas Velocity Map from Sectors-binned Cube	2d_map
gas-vdisp_sectors_recom-comp	Recommended-component Ionised Gas Velocity Dispersion Map from Sectors-binned Cube	2d_map
OII3728_sectors_recom-comp	Recommended-component Line Emission Map from Sectors-binned Cube: [OII] (3726Å+3729Å)	2d_map
Halpha_sectors_recom-comp	Recommended-component Line Emission Map from Sectors-binned Cube: Hα	2d_map
Hbeta_sectors_recom-comp	Recommended-component Line Emission Map from Sectors-binned Cube: Hβ	2d_map
OIII5007_sectors_recom-comp	Recommended-component Line Emission Map from Sectors-binned Cube: [OIII] (5007Å)	2d_map
OI6300_sectors_recom-comp	Recommended-component Line Emission Map from Sectors-binned Cube: [OI] (6300Å)	2d_map
NII6583_sectors_recom-comp	Recommended-component Line Emission Map from Sectors-binned Cube: [NII] (6583Å)	2d_map
SII6716_sectors_recom-comp	Recommended-component Line Emission Map from Sectors-binned Cube: [SII] (6716Å)	2d_map
SII6731_sectors_recom-comp	Recommended-component Line Emission Map from Sectors-binned Cube: [SII] (6731Å)	2d_map
extinct-corr_adaptive_1-comp	Extinction correction map from 1-component adaptive-binned Hα/Hβ flux ratio	2d_map
sfr-mask_adaptive_1-comp	Star formation mask map from 1-component adaptive-binned emission line ratios	2d_map
sfr_adaptive_1-comp	Star formation rate map from 1-component adaptive-binned Hα flux	2d_map
sfr-dens_adaptive_1-comp	Star formation rate surface density from 1-component adaptive-binned Hα flux	2d_map
extinct-corr_adaptive_recom-comp	Extinction correction map from recommended-component adaptive-binned Hα/Hβ flux ratio	2d_map
sfr-mask_adaptive_recom-comp	Star formation mask map from recommended-component adaptive-binned emission line ratios	2d_map
sfr_adaptive_recom-comp	Star formation rate map from recommended-component adaptive-binned Hα flux	2d_map
sfr-dens_adaptive_recom-comp	Star formation rate surface density from recommended-component adaptive-binned Hα flux	2d_map
extinct-corr_sectors_1-comp	Extinction correction map from 1-component sectors-binned Hα/Hβ flux ratio	2d_map
sfr-mask_sectors_1-comp	Star formation mask map from 1-component sectors-binned emission line ratios	2d_map
sfr_sectors_1-comp	Star formation rate map from 1-component sectors-binned Hα flux	2d_map
sfr-dens_sectors_1-comp	Star formation rate surface density from 1-component sectors-binned Hα flux	2d_map
extinct-corr_sectors_recom-comp	Extinction correction map from recommended-component sectors-binned Hα/Hβ flux ratio	2d_map
sfr-mask_sectors_recom-comp	Star formation mask map from recommended-component sectors-binned emission line ratios	2d_map
sfr_sectors_recom-comp	Star formation rate map from recommended-component sectors-binned Hα flux	2d_map
sfr-dens_sectors_recom-comp	Star formation rate surface density from recommended-component sectors-binned Hα flux	2d_map
extinct-corr_annular_2-comp	Extinction correction map from 2-component annular-binned Hα/Hβ flux ratio	2d_map
sfr-mask_annular_2-comp	Star formation mask map from 2-component annular-binned emission line ratios	2d_map
sfr_annular_2-comp	Star formation rate map from 2-component annular-binned Hα flux	2d_map
sfr-dens_annular_2-comp	Star formation rate surface density from 2-component annular-binned Hα flux	2d_map

If you have additional product names that aren’t covered by this list, please contact us by following this link and choosing to Add new data.

description(required=True, type=string, max_limit=500): A succinct paragraph describing the product.

documentation(required=False, type=string, max_limit=1000): If you would like formatted text to appear in the Schema Browser, you must supply a file in the docs/ directory. The file can be either plain text or markdown-formatted text (see Formatting for more info). This field is set to the name of the file. If you do not wish to supply documentation for a particular group, leave this entry blank. Remember that the documentation mentioned here is uneditable once the data are released. Documentation intended for public use and/or subject to change (e.g., with detailed descriptions of analysis or version update information) should be maintained by the survey teams in Data Central’s Documentation portal: Documentation Central.

group_name(required=True, type=string, max_limit=100): The name of the group (this must match a group name from the <survey>_<datarelease>_group_meta.txt file above).

version(required=True, type=string, max_limit=100): Product version as defined by the team e.g., v1.8.

contact(required=True, type=string, max_limit=500): Format as: John Smith <john.smith@institute.org>.

<survey>_<datarelease>_group_meta.txt

To aid the user in finding data products, a grouping is applied in the Schema Browser and the SOV (i.e., all IFS-derived spectra can be grouped under ‘Spectra’). This file describes the groups you would like to register, and will be used to populate the Schema Browser and categorize data products in the SOV.

Please provide a single pipe-delimited .txt file containing an entry (row) for each group, in the following format:

name	pretty_name
cube	Cubes
stelkin	Stellar Kinematics

Please name this file: <survey>_<datarelease>_group_meta.txt, replacing <survey> and <datarelease> with the survey name and datarelease name, respectively: e.g., sami_dr2_group_meta.txt.

<survey>_<datarelease>_group_meta.txt

This file should contain the following columns:

name(required=True, type=string, max_limit=100): Group name. Use only alphanumeric characters. This must be unique per data release.

pretty_name(required=True, type=string, max_limit=500): A human-readable version of the group name. This can contain any characters (up to the character limit).

<survey>_<datarelease>_product_ingestion.txt

This file will link each data product with the relevant source from your input catalogue (which was added in the Source Catalogue Identification). This file should include an entry (row) for every file you wish to ingest into Data Central.

facility_name	data_product_name	file_name	rel_file_path	source_name	is_best
sami	OIII5007_annular_2-comp	99428_OIII5007_annular_2-comp.fits	99428/99428_OIII5007_annular_2-comp.fits	99428	TRUE

Please name this file: <survey>_<datarelease>_product_ingestion_meta.txt, replacing <survey> and <datarelease> with the survey name and datarelease name, respectively: e.g., sami_dr2_product_ingestion_meta.txt.

<survey>_<datarelease>_product_ingestion.txt

This file should contain the following columns:

facility_name(required=True, type=string, max_limit=100): The name of the facility (this must match a facility name from the <survey>_<datarelease>_facility_meta.txt file above).

data_product_name(required=True, type=string, max_limit=100): The name of the data product (this must match a data product name from the <survey>_<datarelease>_product_meta.txt file).

file_name(required=True, type=string, max_limit=1000): The filename of the data product you will be providing.

rel_file_path(required=True, type=string, max_limit=100): The relative path of the file. e.g., <source_name>/product1.fits. See Directory structure.

source_name(required=True, type=string, max_limit=100): The source name as provided in your source catalogue (see Source Catalogue Identification).

is_best(required=False, type=bool, default=True): You may associate multiple IFS with a single product, and define ONE of those products as the best (add a description of this in your <survey>_<datarelease>_product_meta.txt file or data release information on Documentation Central). The best product for a source will be highlighted in the SOV as such.