Jekyll2026-07-15T04:52:24+00:00https://dawn-cph.github.io/dja/feed.xmlDJAThe DAWN JWST Archive, Science-ready public JWST data productsALMA archive data mining with ECOGAL2025-11-20T14:00:32+00:002025-11-20T14:00:32+00:00https://dawn-cph.github.io/dja/blog/2025/11/20/ecogal-dja-showcase imaging ALMA catalog release demo

(This page is auto-generated from the Jupyter notebook ecogal-dja-showcase.ipynb.)

Short description of the project:

ECOGAL (ECOology for Galaxies using ALMA archive and Legacy surveys) is an ALMA data-mining effort that uniformly reduces archival data, creates science-ready ALMA images, and links them to JWST/HST legacy datasets in well-studied survey fields.

This notebook provides an introduction to the ECOGAL catalogue, including how to query sources by position and retrieve the summary plots available for ALMA-detected galaxies with DJA spectra. The catalogue released with this post covers galaxies in the three ALMA-accessible CANDELS fields: COSMOS, GOODS-S, and UDS.

  • This notebook was tested on python 3.12 version
  • Some functions that are used in this notebook can be installed from : https://github.com/mjastro/ecogal
    • or on terminal:

    python -m pip install git+https://github.com/mjastro/ecogal.git

Additional documentation will be released soon. A complete description of the ALMA data reduction and catalogue construction is provided in Lee et al. (2025) [arXiv:2511.20751], and should be cited when using the ECOGAL data products. Users should also cite the appropriate survey references (including ALMA project IDs) when making use of the DJA data products.

%matplotlib inline

# libraries to install
if 0:
    !python -m pip install git+https://github.com/mjastro/ecogal.git
    !pip install tabulate
    !pip install git+https://github.com/karllark/dust_attenuation.git

# setting the libraries

import ecogal

import numpy as np
import astropy
import os,sys
from astropy.coordinates import SkyCoord
from astropy import units as u
from astropy.table import Table
from astropy.utils.data import download_file
import astropy.constants as const

import matplotlib as mpl
import cmasher as cmr
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
from astropy.coordinates import Angle


import shapely
from shapely import Point, Polygon


import warnings
from astropy.io import fits
from astropy.wcs import WCS
warnings.filterwarnings('ignore')

import pandas as pd


# cosmology
from astropy.cosmology import FlatLambdaCDM
cosmo = FlatLambdaCDM(H0=70, Om0=0.3, Tcmb0=2.725)


CACHE_DOWNLOADS = True


print(f'astropy version: {astropy.__version__}')

astropy version: 7.1.1

# Set plotting style
mpl.rcParams['axes.linewidth'] = 2
mpl.rcParams['axes.labelsize'] = 20
mpl.rcParams.update({'font.family':'serif'})
mpl.rcParams.update({'xtick.major.pad': '7.0'})
mpl.rcParams.update({'xtick.major.size': '7.5'})
mpl.rcParams.update({'xtick.major.width': '1.5'})
mpl.rcParams.update({'xtick.minor.pad': '7.0'})
mpl.rcParams.update({'xtick.minor.size': '3.5'})
mpl.rcParams.update({'xtick.minor.width': '1.0'})
mpl.rcParams.update({'ytick.major.pad': '7.0'})
mpl.rcParams.update({'ytick.major.size': '7.5'})
mpl.rcParams.update({'ytick.major.width': '1.5'})
mpl.rcParams.update({'ytick.minor.pad': '7.0'})
mpl.rcParams.update({'ytick.minor.size': '3.5'})
mpl.rcParams.update({'ytick.minor.width': '1.0'})
mpl.rcParams.update({'xtick.labelsize':14})
mpl.rcParams.update({'ytick.labelsize':14})

mpl.rcParams.update({'xtick.direction':'in'})
mpl.rcParams.update({'ytick.direction':'in'})

mpl.rcParams.update({'axes.labelsize' :18})

# to get DJA spectra information

from urllib import request

####Needed to load spectra
import msaexp
import msaexp.spectrum

#to get DJA slit information

import grizli
from grizli import utils


print(f'grizli version: {grizli.__version__}')


grizli version: 1.13.2

Catalogue data exploration

Read the table

# prior catalogue which includes all sources with flux constraints (including non-detection) based on the source positions determined by JWST/HST detection

version ='v1' #initial data release

URL_PREFIX = "https://s3.amazonaws.com/alma-ecogal/dr1"
file_cat = "ecogal_all_priors_"+version+".csv"

if 0:
    # the latest zenodo (frozen) catalogue is available here (TBD)
    # this include blind catalogue, and detection catalogue
    URL_PREFIX = "https://zenodo.org/records/XXX"

table_url = f"{URL_PREFIX}/catalogue/{file_cat}"

tab = utils.read_catalog(download_file(table_url, cache=CACHE_DOWNLOADS), format='csv')

Column descriptions

columns_url = f"{URL_PREFIX}/catalogue/ecogal_{version}.columns.csv"
tab_columns = utils.read_catalog(download_file(columns_url, cache=CACHE_DOWNLOADS), format='csv')

# Set column metadata
for row in tab_columns:
    col = row['column']
    if row['unit'] != '--':
        tab[col].unit = row['unit']
    if row['description'] != '--':
        tab[col].description = row['description']

tab.info()

<GTable length=258455>
       name         dtype      unit                                                                                                     description                                                                                                     class     n_bad 
------------------ ------- ------------ ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ------------ ------
         projectID   str14                                                                                                                                                                                                           ALMA project ID       Column      0
       target_alma   str23                                                                                           ALMA target names without space;if original ALMA program has a space for the target name this column does not include the space       Column      0
  small_mosaic_idx   int64                                                                                                                                                                   mosaic identifier if it was obtained in the mosaic mode       Column      0
             field    str6                                                                                                                                                                                                        legacy field names       Column      0
         id_ecogal   int64                                                                                                                                                                                                          ecogal parant ID       Column      0
            id_new   str13                                                                                                                                                                            ecogal parent full ID including the field name       Column      0
         frequency float64          GHz                                                                                                                                                                                           observed frequency       Column      0
              band    str2                                                                                                                                                                                                        observed ALMA band       Column      0
          beam_maj float64       arcsec                                                                                                                                                                        synthesized beam major axis in arcsec       Column      0
             frame    str4                                                                                                                                                                                                               image frame       Column      0
         RA_parent float64          deg                                                                                                                                                                  prior position from JWST/HST (RA) in degree       Column      0
        Dec_parent float64          deg                                                                                                                                                                 prior position from JWST/HST (DEC) in degree       Column      0
      RA_peak_alma float64          deg                                                                                                                                                                            ALMA peak position (RA) in degree       Column      0
     Dec_peak_alma float64          deg                                                                                                                                                                            ALMA pak position (DEC) in degree       Column      0
  separation_prior float64       arcsec                                                                                                                                                     position offset between ALMA peak and the prior position       Column      0
         flux_peak float64           Jy                                                                                                                                                                                         peak ALMA flux in Jy       Column      0
             noise float64           Jy                                                                                                                                         1-sigma noise level (after correcting for the primary beam response)       Column      0
                sn float64                                                                                                                                                                                                                  peak SNR       Column      0
         flux_aper float64           Jy                                                                                                                                                                            flux from the aperture photometry MaskedColumn   1707
        eflux_aper float64           Jy                                                                                                                                                                      flux error from the aperture photometry       Column      0
   flux_peak_imfit float64           Jy                                                                                                                                                           peak flux from the 2D Gaussian fitting using imfit       Column      0
        flux_imfit float64           Jy                                                                                                                                                                flux from the 2D Gaussian fitting using imfit       Column      0
       eflux_imfit float64           Jy                                                                                                                                                          flux error from the 2D Gaussian fitting using imfit       Column      0
         fac_pbcor float64                                                                                                                                                              primary beam response at the position of the source;1=center       Column      0
       id_3dhst_v4 float64                                                                                                                                                                                3D-HST(ver4.0) catalogue id when available MaskedColumn 142476
        ecogal_ver    str4                                                                                                                                                                                                            ecogal version       Column      0
    zsp_best_avail float64                                                                                                                                                                                     best available spectroscopic redshift       Column      0
   zsp_best_survey    str9                                                                                                                                                                                        selected spectroscopic survey name       Column      0
             z_ver    str4                                                                                                                                                                                                           DJA MSA version       Column      0
             objid float64                                                                                                                                                                                                  Unique source identifier MaskedColumn 248862
             srcid float64                                                                                                                                                                                                   Source ID from APT plan MaskedColumn 248862
              file   str55                                                                                                                                                                                                              DJA filename MaskedColumn 248862
           grating    str5                                                                                                                                                                                                           NIRSpec grating MaskedColumn 248862
         file_phot   str44                                                                                                                                                                                   Filename of the DJA photometric catalog MaskedColumn 249835
           id_phot float64                                                                                                                                                                                 ID number in the DJA photometric cadtalog MaskedColumn 249835
             valid    str5                                                                                                                                                                            Redshift matches best z from visual inspection       Column      0
       z_phot_eazy float64                                                                                                                                                          photometric redshift from eazypy combining 3D-HST+DJA photometry       Column      0
             restU float64                                                                                                                                                        flux density of the rest-frame U band from the photoz (eazypy) fit       Column      0
         restU_err float64                                                                                                                                                  flux density error of the rest-frame U band from the photoz (eazypy) fit       Column      0
             restV float64                                                                                                                                                        flux density of the rest-frame V band from the photoz (eazypy) fit       Column      0
         restV_err float64                                                                                                                                                        flux density of the rest-frame V band from the photoz (eazypy) fit       Column      0
             restJ float64                                                                                                                                                        flux density of the rest-frame J band from the photoz (eazypy) fit       Column      0
         restJ_err float64                                                                                                                                                        flux density of the rest-frame J band from the photoz (eazypy) fit       Column      0
           mass_ez float64      solMass                                                                                                                                                                    stellar mass from the photoz (eazypy) fit       Column      0
            sfr_ez float64 solMass / yr                                                                                                                                                             star-formation rate from the photoz (eazypy) fit       Column      0
             Av_ez float64          mag                                                                                                                                                                                   Av the photoz (eazypy) fit       Column      0
             lmass float64      solMass                                                                                                                                                     stellar mass in log from FAST++ fit (spec-z source only) MaskedColumn      8
         l68_lmass float64      solMass                                                                                                                               68% lower boundary of stellar mass in log from FAST++ run (spec-z source only) MaskedColumn      8
         u68_lmass float64      solMass                                                                                                                               68% upper boundary of stellar mass in log from FAST++ run (spec-z source only) MaskedColumn      8
              lsfr float64 solMass / yr                                                                                                                                              star-formation rate in log from FAST++ fit (spec-z source only) MaskedColumn      8
          l68_lsfr float64 solMass / yr                                                                                                                        68% lower boundary of star-formation rate in log from FAST++ fit (spec-z source only) MaskedColumn      8
          u68_lsfr float64 solMass / yr                                                                                                                        68% upper boundary of star-formation rate in log from FAST++ fit (spec-z source only) MaskedColumn      8
         file_alma  str137                                                                                                                                                                       ECOGAL ALMA fits file name (primary beam corrected)       Column      0
           RA_bdsf float64          deg                                                                                                                                                                        ALMA position(RA) based on PYBDSF fit MaskedColumn 255029
          DEC_bdsf float64          deg                                                                                                                                                                       ALMA position(Dec) based on PYBDSF fit MaskedColumn 255029
  Total_flux_pbcor float64           Jy                                                                                                                                     Total flux based on PYBDSF fit (corrected for the primary beam response) MaskedColumn 255029
E_Total_flux_pbcor float64           Jy                                                                                                                               Total flux error based on PYBDSF fit (corrected for the primary beam response) MaskedColumn 255029
             pbfac float64                                                                                                                                          ALMA primary beam response at the position of the detected source;1=phase center MaskedColumn 255029
         Peak_flux float64           Jy                                                                                                                                                 Peak flux from PYBDSF fit (before the primary beam response) MaskedColumn 255029
       E_Peak_flux float64           Jy                                                                                                                                           Peak flux error from PYBDSF fit (before the primary beam response) MaskedColumn 255029
               Maj float64          deg                                                                                                                                                                     the FWHM of the major axis of the source MaskedColumn 255029
             E_Maj float64          deg                                                                                                                                                the 1-sigma error on the FWHM of the major axis of the source MaskedColumn 255029
               Min float64          deg                                                                                                                                                                     the FWHM of the minor axis of the source MaskedColumn 255029
             E_Min float64          deg                                                                                                                                                the 1-sigma error on the FWHM of the minor axis of the source MaskedColumn 255029
                PA float64          deg                                                                                                                                    the position angle of the major axis of the source measured east of north MaskedColumn 255029
              E_PA float64          deg                                                                                                                                      the 1-sigma error on the position angle of the major axis of the source MaskedColumn 255029
     Maj_img_plane float64          deg                                                                                                                                                  the FWHM of the major axis of the source in the image plane MaskedColumn 255029
   E_Maj_img_plane float64          deg                                                                                                                                         the 1-sigma error of the major axis of the source in the image plane MaskedColumn 255029
     Min_img_plane float64          deg                                                                                                                                                  the FWHM of the minor axis of the source in the image plane MaskedColumn 255029
   E_Min_img_plane float64          deg                                                                                                                                         the 1-sigma error of the minor axis of the source in the image plane MaskedColumn 255029
      PA_img_plane float64          deg                                                                                                                 the position angle in the image plane of the major axis of the source measured east of north MaskedColumn 255029
    E_PA_img_plane float64          deg                                                                                                                  the 1-sigma error of the image plane of the major axis of the source measured east of north MaskedColumn 255029
    Isl_Total_flux float64           Jy                                                                                         the total integrated Stokes I flux density of the island in which the source is located (not primary beam corrected) MaskedColumn 255029
  E_Isl_Total_flux float64           Jy                                                                                                                     the 1-sigma error on the total flux density of the island in which the source is located MaskedColumn 255029
           Isl_rms float64    Jy / beam                                                                                                                                    the average background rms value of the island (derived from the rms map) MaskedColumn 255029
            S_Code    str1              a code that defines the source structure; ‘S’ = a single-Gaussian source that is the only source in the island; ‘C’ = a single-Gaussian source in an island with other sources;'M’ = a multi-Gaussian source MaskedColumn 255029
   Separation_bdsf float64       arcsec                                                                                                                                           offset between the optical counterpart (matched within 0.8 arcsec) MaskedColumn 255029

zphot-zspec

### -- getting the unique source and spec-z sources
con_dup = np.array(tab.to_pandas()['id_new'].duplicated())
tab0 = tab[~con_dup]
len(tab0)

128125

con_z = tab0['zsp_best_avail']>0
con_z &= tab0['z_phot_eazy']>0

con_dja = tab0['zsp_best_survey']=='dja'
con_dja_z = np.logical_and(con_dja,con_z)

plt.scatter(tab0[con_z]['zsp_best_avail'], tab0[con_z]['z_phot_eazy'], s=1, zorder=100, label='All spec-z')
plt.scatter(tab0[con_dja_z]['zsp_best_avail'], tab0[con_dja_z]['z_phot_eazy'], marker='s',facecolor='None',edgecolor='grey',alpha=0.5, label='DJA spec-z')

plt.xlabel(r'$z_{\rm spec}$')
plt.ylabel(r'$z_{\rm phot}$')
plt.legend()

<matplotlib.legend.Legend at 0x331825df0>

png

Query if there is any ALMA coverage given the position

  • ALMA/ECOGAL metadata includes information of individual ALMA images such as regions, pixel scales, phase center position, etc.

  • This will allow you to check the ALMA/ECOGAL coverage for your source of interest.

# A complete version of the metadata

version='v1'
meta_file = "ecogal_"+version+"_metadata.fits"

table_url = f"{URL_PREFIX}/ancillary/{meta_file}"
meta = utils.read_catalog(download_file(table_url, cache=CACHE_DOWNLOADS), format='fits')
meta[:3]
GTable length=3
file_almaversionsimplebitpixnaxisnaxis1naxis2naxis3extendbscalebzerobmajbminbpabtypeobjectbunitequinoxradesyslonpolelatpolepc1_1pc2_1pc3_1pc1_2pc2_2pc3_2pc1_3pc2_3pc3_3ctype1crval1cdelt1crpix1cunit1ctype2crval2cdelt2crpix2cunit2ctype3crval3cdelt3crpix3cunit3pv2_1pv2_2restfrqspecsysaltrvalaltrpixvelreftelescopobserverdate-obstimesysobsraobsdecobsgeo-xobsgeo-yobsgeo-zinstrumedistancempiprocschnchnksmemreqmemavailuseweighdateoriginalmaidis_mosaicbandfootprintreleaseis_availablera_centerdec_centernoise_fitnoise_totFoV_sigma
bytes137bytes4boolint64int64int64int64int64boolfloat64float64float64float64float64bytes9bytes26bytes7float64bytes4float64float64float64float64float64float64float64float64float64float64float64bytes8float64float64float64bytes3bytes8float64float64float64bytes3bytes4float64float64float64bytes2float64float64float64bytes4float64float64int64bytes4bytes15bytes26bytes3float64float64float64float64float64bytes4float64int64int64float64float64boolbytes26bytes30bytes14boolint64bytes461bytes3boolfloat64float64float64float64float64
2011.0.00064.S___concat_all_6_AzTEC-3_0_b7_cont_noninter2sig.image.pbcor.fitsv1.0True-323160016001True1.00.00.00020058131880230.0001579564147525-53.50122070312IntensityAzTEC-3Jy/beam2000.0FK5180.02.5868333363111.00.00.00.01.00.00.00.01.0RA---SIN150.0890000048-2.777777777778e-05801.0degDEC--SIN2.5868333363112.777777777778e-05801.0degFREQ296763686486.815646751913.561.0Hz0.00.0296763686486.8LSRK-0.01.0257ALMAriechers2012-04-11T01:22:13.632000UTC150.08900000482.5868333363112225142.180269-5440307.370349-2481029.851874ALMA0.0110.12359619150.23436False2023-11-14T12:19:42.442402CASA 6.5.6-22 CASAtools:v1.0.02011.0.00064.SFalse7((150.088583,2.582778),(150.085914,2.584167),(150.084940,2.587250),(150.086331,2.589917),(150.089417,2.590889),(150.092086,2.589500),(150.093060,2.586417),(150.091669,2.583750),(150.088583,2.582778))N/ATrue150.08900000482.58683333631100254.46259777172499e-055.5606829846510664e-058.332485222950787
2011.0.00064.S___concat_all_6_AzTEC-3_1_b7_cont_noninter2sig.image.pbcor.fitsv1.0True-323160016001True1.00.00.00020078808069230.0001580240825812-53.53054428101IntensityAzTEC-3Jy/beam2000.0FK5180.02.5885277779321.00.00.00.01.00.00.00.01.0RA---SIN150.0868750002-2.777777777778e-05801.0degDEC--SIN2.5885277779322.777777777778e-05801.0degFREQ296763687761.715646752334.741.0Hz0.00.0296763687761.7LSRK-0.01.0257ALMAriechers2012-04-11T01:22:54.720000UTC150.08687500022.5885277779322225142.180269-5440307.370349-2481029.851874ALMA0.0110.12359619150.1451False2023-11-14T13:30:14.601414CASA 6.5.6-22 CASAtools:v1.0.02011.0.00064.SFalse7((150.086458,2.584472),(150.083789,2.585861),(150.082815,2.588944),(150.084206,2.591611),(150.087292,2.592583),(150.089961,2.591194),(150.090935,2.588111),(150.089544,2.585444),(150.086458,2.584472))N/ATrue150.08687500022.58852777793200066.49289000021855e-050.000117142153612803678.33248518715434
2011.0.00097.S___concat_all_10_COSMOSLowz_64_29_b7_cont_noninter2sig.image.pbcor.fitsv1.0True-323184418441True1.00.00.00014458558625650.000138595170444932.57455062866IntensityCOSMOSLowz_64Jy/beam2000.0FK5180.02.1937788888891.00.00.00.01.00.00.00.01.0RA---SIN150.0951-2.777777777778e-05923.0degDEC--SIN2.1937788888892.777777777778e-05923.0degFREQ341959206917.115956638554.121.0Hz0.00.0341959206917.1LSRK-0.01.0257ALMAnscoville2012-04-22T02:13:22.032000UTC150.09512.1937788888892225142.180269-5440307.370349-2481029.851874ALMA0.0--------False2022-01-18T10:06:48.750999CASA 5.6.1-82011.0.00097.STrue7((150.094822,2.190251),(150.092487,2.191390),(150.091570,2.194057),(150.092709,2.196390),(150.095378,2.197307),(150.097713,2.196168),(150.098630,2.193501),(150.097491,2.191168),(150.094822,2.190251))dr1True150.09512.19377888888899970.000139687380281118630.000141854310641065247.231210572315804

Use ecogal function : query metata data based on the coordinates

Use visualcheck.get_footprint function.

import ecogal.visualcheck as visualcheck

# get the metadata via DJA
ra, dec = 34.41887, -5.21965
fp,_ = visualcheck.get_footprint(ra,dec)

There are #15 ALMA projects overlapping

fp['file_alma','almaid','object','band']
GTable length=15
file_almaalmaidobjectband
bytes137bytes14bytes26int64
2012.1.00245.S__all_SXDF-NB2315-2_b7_cont_noninter2sig.image.pbcor.fits2012.1.00245.SSXDF-NB2315-27
2012.1.00245.S__all_SXDF-NB2315-3_b7_cont_noninter2sig.image.pbcor.fits2012.1.00245.SSXDF-NB2315-37
2013.1.00742.S__all_SXDF-B3-NB2315-FoV1_b3_cont_noninter2sig.image.pbcor.fits2013.1.00742.SSXDF-B3-NB2315-FoV13
2013.1.00781.S__all_SXDS-AzTEC23_b6_cont_noninter2sig.image.pbcor.fits2013.1.00781.SSXDS-AzTEC236
2015.1.00442.S__all_SXDS-AzTEC28_b6_cont_noninter2sig.image.pbcor.fits2015.1.00442.SSXDS-AzTEC286
2015.1.01074.S__all_UDSp_17_b7_cont_noninter2sig.image.pbcor.fits2015.1.01074.SUDSp_177
2015.1.01528.S__all_UDS.0113_b7_cont_noninter2sig.image.pbcor.fits2015.1.01528.SUDS.01137
2016.1.00434.S__all_UDS.0113_b7_cont_noninter2sig.image.pbcor.fits2016.1.00434.SUDS.01137
2017.1.00562.S__all_NB2315_b3_cont_noninter2sig.image.pbcor.fits2017.1.00562.SNB23153
2017.1.00562.S__all_NB2315_b6_cont_noninter2sig.image.pbcor.fits2017.1.00562.SNB23156
2017.1.00562.S__all_U4-16795_b9_cont_noninter2sig.image.pbcor.fits2017.1.00562.SU4-167959
2017.1.01027.S__all_U4-16504_b7_cont_noninter2sig.image.pbcor.fits2017.1.01027.SU4-165047
2017.1.01027.S__all_U4-16795_b7_cont_noninter2sig.image.pbcor.fits2017.1.01027.SU4-167957
2019.1.00337.S__all_AS2UDS0113.1_b3_cont_noninter2sig.image.pbcor.fits2019.1.00337.SAS2UDS0113.13
2021.1.00705.S__all_UDS.0113_b4_cont_noninter2sig.image.pbcor.fits2021.1.00705.SUDS.01134

Get ALMA/ECOGAL cutouts

Method 1. Using ecogal

Use ecogal.pbcor.show_all_cutouts function.

import ecogal.pbcor as ecogal_plot

[1] JWST RGB + ALMA cutout

# an example of many ALMA coverage
ra, dec = 34.41887, -5.21965

# this example can take a while because it downloads a lot of fits files
summary_cutouts = ecogal_plot.show_all_cutouts(ra,dec)

N=15
2013.1.00742.S__all_SXDF-B3-NB2315-FoV1_b3       b3   157x 157  0.04  dx=7.57"
2017.1.00562.S__all_NB2315_b3                    b3   313x 313  0.02  dx=7.57"
2019.1.00337.S__all_AS2UDS0113.1_b3              b3    63x  63  0.10  dx=9.84"
2021.1.00705.S__all_UDS.0113_b4                  b4    33x  33  0.20  dx=6.65"
2015.1.00442.S__all_SXDS-AzTEC28_b6              b6   157x 157  0.04  dx=8.32"
2013.1.00781.S__all_SXDS-AzTEC23_b6              b6   313x 313  0.02  dx=8.33"
2017.1.00562.S__all_NB2315_b6                    b6   313x 313  0.02  dx=4.83"
2016.1.00434.S__all_UDS.0113_b7                  b7   157x 157  0.04  dx=6.65"
2012.1.00245.S__all_SXDF-NB2315-3_b7             b7   313x 313  0.02  dx=9.65"
2015.1.01074.S__all_UDSp_17_b7                   b7   313x 313  0.02  dx=9.75"
2015.1.01528.S__all_UDS.0113_b7                  b7   313x 313  0.02  dx=6.66"
2012.1.00245.S__all_SXDF-NB2315-2_b7             b7   313x 313  0.02  dx=0.15"
2017.1.01027.S__all_U4-16504_b7                  b7    63x  63  0.10  dx=9.77"
2017.1.01027.S__all_U4-16795_b7                  b7    63x  63  0.10  dx=0.42"
2017.1.00562.S__all_U4-16795_b9                  b9   313x 313  0.02  dx=0.06"

png

[2] Ready-made summary file

There are some summary png files in the repository made for sources with ALMA detection and DJA spectra (~120 unique sources in total with the first data release), which can be queried by the source position.

  • the default searching area is 0.4 arcsec, that you can change with r_search
ra, dec = 150.14325, 2.35599	
_ = visualcheck.get_summary(ra,dec, r_search=0.5)

There are #10 ALMA projects overlapping
There are 9 ECOGAL+DJA cross-match!
https://s3.amazonaws.com/alma-ecogal/dr1/pngs/ecogal__0_all_filters_COSMOS.60520.png
A source found at a distance of = 0.13 arcsec


ra, dec = 34.27751, -5.22819
_ = visualcheck.get_summary(ra,dec)

There are #1 ALMA projects overlapping
There are 1 ECOGAL+DJA cross-match!
https://s3.amazonaws.com/alma-ecogal/dr1/pngs/ecogal__0_all_filters_UDS.104633.png
A source found at a distance of = 0.13 arcsec

Method 2: Getting cutouts and footprint via DJA API

There are three different output modes:

  • output=footprint : footprint for ALMA coverage
  • output=csv : metadata
  • output=cutout : making a cutout image
    • for cutout module: specify ALMA file names. The file name is available in the ECOGAL catalogue or from the metadata, and the column name is file_alma

footprint mode

https://grizli-cutout.herokuapp.com/ecogal?output=footprint&ra=34.48016&dec=-5.11252

cutout mode

Given the ALMA file name of 2015.1.01528.S__all_UDS.0424_b7_cont_noninter2sig.image.pbcor.fits

https://grizli-cutout.herokuapp.com/ecogal?output=cutout&sx=3.&cutout_size=2.0&ra=34.48016&dec=-5.11252&file_alma=2015.1.01528.S__all_UDS.0424_b7_cont_noninter2sig.image.pbcor.fits

from IPython.display import display, Markdown, Latex

# getting the summary of the footprint for a given ra, dec
ra, dec = 34.48016,-5.11252
cord = SkyCoord(ra,dec, unit=(u.degree, u.degree))
ara = tab['RA_peak_alma']
adec = tab['Dec_peak_alma']
acord = SkyCoord(ara, adec, unit=(u.degree, u.degree))
#search for the matching source within 0.1 arcsec
con_pos = acord.separation(cord).arcsec<0.15
ecotb=tab[con_pos]


#  see also the description in the https://dawn-cph.github.io/dja/blog/2025/05/01/nirspec-merged-table-v4/
#  
cutout_URL = f"https://grizli-cutout.herokuapp.com/ecogal?output=footprint&ra={ra}&dec={dec}"

ecotb['Thumb'] = [
    "<img src=\"{0}\" height=200px>".format(
        cutout_URL.format(ra,dec)
    )
    for row in ecotb
]

df = ecotb['projectID','target_alma','id_new','band','beam_maj','sn','separation_prior','zsp_best_avail','z_phot_eazy','Thumb','file_alma'].to_pandas()

display(Markdown(df.to_markdown()))
  projectID target_alma id_new band beam_maj sn separation_prior zsp_best_avail z_phot_eazy Thumb file_alma
0 2023.1.01520.S 0424.0 UDS.105062 b4 0.69 7.08 0.09 3.5433 3.47313 2023.1.01520.S__all_0424.0_b4_cont_noninter2sig.image.pbcor.fits
1 2015.1.01528.S UDS.0424 UDS.105062 b7 0.21 19.15 0.05 3.5433 3.47313 2015.1.01528.S__all_UDS.0424_b7_cont_noninter2sig.image.pbcor.fits
2 2013.1.00781.S SXDS-AzTEC28 UDS.105062 b6 0.32 18.72 0.06 3.5433 3.47313 2013.1.00781.S__all_SXDS-AzTEC28_b6_cont_noninter2sig.image.pbcor.fits

Access the ALMA/ECOGAL image fits files

  • All fits files (primary beam corrected) are available from the DJA repository (on AWS server) and a frozen version will be available on Zenodo.
#getting the corresponding file name from the catalogue
#some times the file name includes special string like "+", which should be parsed to download the fits file

idx = 2
file_alma = ecotb['file_alma'][idx]
encoded_filename = file_alma.replace("+", "%2B")

fits_URL = "https://s3.amazonaws.com/alma-ecogal/dr1/pbcor/"

almafits = fits.open(
    download_file(str(os.path.join(fits_URL, encoded_filename)), cache=True)
                    )

img = almafits[0].data[0]
hdr = almafits[0].header
wcs_alma = WCS(hdr)

gid = tab[con_pos]['id_new'][idx]
band = tab[con_pos]['band'][idx]
zgal = tab[con_pos]['zsp_best_avail'][idx]
lmass = np.log10(tab[con_pos]['mass_ez'][idx])
sfr = tab[con_pos]['sfr_ez'][idx]

#get the noise of the map (after pb-correction)
noise = tab[con_pos]['noise'][idx]

##################
## plotting; think about adding a function
##################

x,y,_= wcs_alma.wcs_world2pix(ra,dec,0,0)
pixsz = np.abs(hdr['CDELT1']*3600)
imsz = 1/pixsz

noise_array = noise*np.arange(4,30,3)
cutout = img[int(y-imsz):int(y+imsz),int(x-imsz):int(x+imsz)]

fig=plt.figure(1,figsize=(5,5))
ax=plt.subplot(111)

ax.imshow(cutout, origin='lower')
ax.contour(cutout, levels = noise_array, colors='white')
ax.set_title(f'{gid},{band}')
ax.text(0.55,0.95, f'z={zgal}', transform=ax.transAxes, c='white')
ax.text(0.55,0.90, f'log(Mstar/Msun)={lmass:.2f}', transform=ax.transAxes, c='white')
ax.text(0.55,0.85, f'sfr={sfr:.2f} Msun/yr', transform=ax.transAxes, c='white')



#get beam size 
bmaj = hdr['BMAJ']*3600
bmin = hdr['BMIN']*3600
pa = hdr['BPA']
theta = Angle(90+pa,'deg') 
be=mpatches.Ellipse((imsz/6.,imsz/6.),bmaj/pixsz,bmin/pixsz,angle=theta.degree,lw=1,facecolor='grey',edgecolor='black',hatch='//')
ax.add_patch(be)


<matplotlib.patches.Ellipse at 0x346f95010>

png

Access the DJA files from the information available in the catalogue

## consider peak SNR>15 and DJA spectra at z>5

con_sn = tab0['sn']>15
con_sn &= tab0['zsp_best_survey']=='dja'
con_sn &= tab0['zsp_best_avail']>5

tab1 = tab0[con_sn]
filename = tab1['file'][0]
zgal = tab1['zsp_best_avail'][0]
filename, zgal

('capers-cos01-v4_prism-clear_6368_52597.spec.fits', 5.8351)

# prepare a line detectionary
# ** line dictionary -- for plot purpose (you can add more)
# ** lines in AA to overplot the emission lines

lam_file = Table.read("""line, wavelength_nm
H-alpha,656.46
H-beta,486.271
H-delta,410.1734
H-gamma,434.0472
Ly-alpha,121.567
[OII],372.71
[OII],372.986
[OII],733.1
[OII],732.0
[NeIII],386.986
[OIII],496.03
[OIII],500.824
[NII],654.986
[NII],658.527
[SII],671.827
[SII],673.267
Pa-alpha,1875
Pa-beta,1282
Pa-gamma,1093.8
Br-beta,2626
HeI,1083.0
[SIII],906.9
[SIII],953.0
[CI],985.0
[PII],1188
[FeII],1257
[FeII],1640      
""", format="csv")

#lam_file = Table.read('opt_emission_lines.csv')

lines_dic = {'Ly-alpha':r'Ly$\alpha$', 'H-beta':r'H$\beta$', 'H-alpha': r'H$\alpha$ + [NII]', 
             'H-delta':r'H$\delta$','H-gamma':r'H$\gamma$', 'Na1D':'NaID',
             'Pa-alpha':r'Pa$\alpha$','Pa-beta':r'Pa$\beta$','Pa-gamma':r'Pa$\gamma$','CaIIK':'Ca II K',
             '[SiVI]':'[SiVI]','H2':r'H$_2$',
             'Br-beta':r'Br$\gamma$','MgI':'MgI','NaI':'NaI','SiI':'SiI','HI':'[FeII]+HI+FeII',#'HI':'HI'-- for grating
             'HeI':'HeI','FeII':'FeII','[FeII]':'[FeII]','[CI]':'[CI]','[PII]':'[PII]',
             '[OII]': '[OII]', '[NII]':'[NII]', '[SII]':'[SII]', '[OIII]':'[OIII]','[NeIII]':'[NeIII]',
             '[SIII]':'[SIII]'}

lam_file
Table length=27
linewavelength_nm
str8float64
H-alpha656.46
H-beta486.271
H-delta410.1734
H-gamma434.0472
Ly-alpha121.567
[OII]372.71
[OII]372.986
[OII]733.1
[OII]732.0
......
Pa-beta1282.0
Pa-gamma1093.8
Br-beta2626.0
HeI1083.0
[SIII]906.9
[SIII]953.0
[CI]985.0
[PII]1188.0
[FeII]1257.0
[FeII]1640.0

set some functions to get DJA spectra

########################################
### Load the DJA spectra information:
########################################

def loadDJAspec(file_fullpath):
    sh = msaexp.spectrum.SpectrumSampler(file_fullpath, err_median_filter=None)
    wave = np.array(sh.spec["wave"])*1e4 ###in AA
    fnu = sh.spec["flux"] ##uJy
    fnu_err = sh.spec["full_err"] ##uJy

    spec_flambda = np.array((fnu*u.uJy/(wave*u.AA)**2*const.c).to(u.erg/u.s/u.cm**2/u.AA)/(u.erg/u.s/u.cm**2/u.AA))
    spec_flambda_err = np.array((fnu_err/(wave*u.AA)**2*const.c).to(u.erg/u.s/u.cm**2/u.AA)/(u.erg/u.s/u.cm**2/u.AA))
    return(sh,wave,spec_flambda,spec_flambda_err)


#################################################
### settting up the DJA spectra information
#################################################

def get_dja_spectra(file, zsp, restframe=False):
    #plotting in the restframe
    
    root = file.split('_')[0]+'/'
    basename = 'https://s3.amazonaws.com/msaexp-nirspec/extractions/'
    fullname = basename+root+file

    print(f'A galaxy at z={zsp}')
    sampled,lam,galaxy,noise = loadDJAspec(fullname)

    #deredshift
    lam0 = lam/(1 + zsp)
    if restframe:
        lam /=(1 + zsp) # Compute approximate restframe wavelength
    

    ####mask areas
    con_mask = noise==0 
    con_mask |= noise>galaxy[lam > 1100].max()
    galaxy_masked = np.ma.masked_where(con_mask, galaxy)


    galaxy0=galaxy_masked
    noise0=noise
    lam0=lam #in AA
    return lam0, galaxy0, noise0



        

lam0, galaxy0, noise0 = get_dja_spectra(filename, zgal, restframe=True)
plt.plot(lam0[lam0>1300],galaxy0[lam0>1300])
plt.xlabel(f'Rest-Wavelength ($\AA$)')
plt.ylabel(f'Flux (erg/s/cm$^2$/$\AA$)')

##adding line identifiers
lines_plotted=[]
ticker_max = galaxy0.max()
text_hi = ticker_max+1e-20
for kk in range(len(lam_file)):
    lam_idx = np.argmin(np.abs(lam_file['wavelength_nm'][kk]*10-lam0))
    
    if lam_file['wavelength_nm'][kk]*10> lam0.min() and lam_file['wavelength_nm'][kk]*10<lam0.max() and noise0[lam_idx]>0:
        plt.axvline(lam_file['wavelength_nm'][kk]*10, linestyle=':', color='darkgrey')
        if lam_file['line'][kk] not in lines_plotted:
            #lines_plotted.append(lam_file['line'][kk])
            if lam_file['line'][kk] not in ['[NII]']:#if PRISM #'[FeII]','FeII'
                plt.text(lam_file['wavelength_nm'][kk]*10+0.01, text_hi*(0.9-0.1*np.mod(kk,3)), lines_dic[lam_file['line'][kk]], rotation='vertical', fontsize=9)


A galaxy at z=5.8351

png

get source information with multiple DJA spectra

# Full DJA table

table_url = "https://s3.amazonaws.com/msaexp-nirspec/extractions/dja_msaexp_emission_lines_v4.4.csv.gz"
tab00 = utils.read_catalog(download_file(table_url, cache=True), format='csv')



con_sn = tab0['sn']>10
con_sn &= tab0['zsp_best_survey']=='dja'
con_sn &= tab0['zsp_best_avail']>4

tab0[con_sn]['file','objid']
GTable length=6
fileobjid
str55float64
gto-wide-uds13-v4_prism-clear_1215_1951.spec.fits172449.0
capers-cos01-v4_prism-clear_6368_52597.spec.fits141884.0
rubies-uds23-v4_prism-clear_4233_166691.spec.fits149974.0
capers-cos04-v4_prism-clear_6368_36571.spec.fits143258.0
rubies-uds1-v4_prism-clear_4233_37108.spec.fits151060.0
gto-wide-uds13-v4_prism-clear_1215_1472.spec.fits162424.0
## check the alternative spectra information

dja_uniq_id = 172449
print(dja_uniq_id)

#search for other dja spectra
msa = tab00[tab00['objid'] == dja_uniq_id]
msa

172449
GTable length=4
filesrcidradecgratingfiltereffexptmnfilesdatasetmsametmsaidmsacnfdithnslitidrootnpixndetwminwmaxwmaxsnsn10flux10err10sn50flux50err50sn90flux90err90xstartystartxsizeysizeslit_papa_v3srcypixprofcenprofsigctimeversionexptimecontchi2doffullchi2line_ariii_7138line_ariii_7138_errline_ariii_7753line_ariii_7753_errline_braline_bra_errline_brbline_brb_errline_brdline_brd_errline_brgline_brg_errline_hbline_hb_errline_hdline_hd_errline_hei_1083line_hei_1083_errline_hei_3889line_hei_3889_errline_hei_5877line_hei_5877_errline_hei_7065line_hei_7065_errline_hei_8446line_hei_8446_errline_heii_4687line_heii_4687_errline_hgline_hg_errline_lyaline_lya_errline_mgiiline_mgii_errline_neiii_3867line_neiii_3867_errline_neiii_3968line_neiii_3968_errline_nev_3346line_nev_3346_errline_nevi_3426line_nevi_3426_errline_niii_1750line_niii_1750_errline_oi_6302line_oi_6302_errline_oiiline_oii_7325line_oii_7325_errline_oii_errline_oiiiline_oiii_1663line_oiii_1663_errline_oiii_4363line_oiii_4363_errline_oiii_4959line_oiii_4959_errline_oiii_5007line_oiii_5007_errline_oiii_errline_pa10line_pa10_errline_pa8line_pa8_errline_pa9line_pa9_errline_paaline_paa_errline_pabline_pab_errline_padline_pad_errline_pagline_pag_errline_pfbline_pfb_errline_pfdline_pfd_errline_pfeline_pfe_errline_pfgline_pfg_errline_siiline_sii_errline_siii_9068line_siii_9068_errline_siii_9531line_siii_9531_errspl_0spl_0_errspl_1spl_10spl_10_errspl_11spl_11_errspl_12spl_12_errspl_13spl_13_errspl_14spl_14_errspl_15spl_15_errspl_16spl_16_errspl_17spl_17_errspl_18spl_18_errspl_19spl_19_errspl_1_errspl_2spl_20spl_20_errspl_21spl_21_errspl_22spl_22_errspl_2_errspl_3spl_3_errspl_4spl_4_errspl_5spl_5_errspl_6spl_6_errspl_7spl_7_errspl_8spl_8_errspl_9spl_9_errzlineline_civ_1549line_civ_1549_errline_h10line_h10_errline_h11line_h11_errline_h12line_h12_errline_h7line_h7_errline_h8line_h8_errline_h9line_h9_errline_haline_ha_errline_hei_6680line_hei_6680_errline_heii_1640line_heii_1640_errline_nii_6549line_nii_6549_errline_nii_6584line_nii_6584_errline_oii_7323line_oii_7323_errline_oii_7332line_oii_7332_errline_sii_6717line_sii_6717_errline_sii_6731line_sii_6731_errline_siii_6314line_siii_6314_errescale0escale1line_ciii_1906line_ciii_1906_errline_niv_1487line_niv_1487_errline_pah_3p29line_pah_3p29_errline_pah_3p40line_pah_3p40_erreqw_ariii_7138eqw_ariii_7753eqw_braeqw_brbeqw_brdeqw_brgeqw_ciii_1906eqw_civ_1549eqw_ha_niieqw_hbeqw_hdeqw_hei_1083eqw_hei_3889eqw_hei_5877eqw_hei_7065eqw_hei_8446eqw_heii_1640eqw_heii_4687eqw_hgeqw_lyaeqw_mgiieqw_neiii_3867eqw_neiii_3968eqw_nev_3346eqw_nevi_3426eqw_niii_1750eqw_niv_1487eqw_oi_6302eqw_oiieqw_oii_7325eqw_oiiieqw_oiii_1663eqw_oiii_4363eqw_oiii_4959eqw_oiii_5007eqw_pa10eqw_pa8eqw_pa9eqw_paaeqw_pabeqw_padeqw_pageqw_pfbeqw_pfdeqw_pfeeqw_pfgeqw_siieqw_siii_9068eqw_siii_9531line_ha_niiline_ha_nii_erreqw_h10eqw_h11eqw_h12eqw_h7eqw_h8eqw_h9eqw_haeqw_hei_6680eqw_nii_6549eqw_nii_6584eqw_oii_7323eqw_oii_7332eqw_sii_6717eqw_sii_6731eqw_siii_6314sn_lineztimeline_ci_9850line_ci_9850_errline_feii_11128line_feii_11128_errline_pii_11886line_pii_11886_errline_feii_12570line_feii_12570_erreqw_ci_9850eqw_feii_11128eqw_pii_11886eqw_feii_12570line_feii_16440line_feii_16440_errline_feii_16877line_feii_16877_errline_brfline_brf_errline_feii_17418line_feii_17418_errline_breline_bre_errline_feii_18362line_feii_18362_erreqw_feii_16440eqw_feii_16877eqw_brfeqw_feii_17418eqw_breeqw_feii_18362validobjidz_bestztypez_prismz_gratingphot_correctionphot_flux_radiusphot_drfile_photid_photphot_mag_autophot_f090w_tot_1phot_f090w_etot_1phot_f115w_tot_1phot_f115w_etot_1phot_f150w_tot_1phot_f150w_etot_1phot_f200w_tot_1phot_f200w_etot_1phot_f277w_tot_1phot_f277w_etot_1phot_f356w_tot_1phot_f356w_etot_1phot_f410m_tot_1phot_f410m_etot_1phot_f444w_tot_1phot_f444w_etot_1phot_Avphot_massphot_restUphot_restVphot_restJz_photphot_LHaphot_LOIIIphot_LOIIgradezgradereviewercommentzrfescaleobs_239_validobs_239_fracobs_239_fluxobs_239_errobs_239_full_errobs_205_validobs_205_fracobs_205_fluxobs_205_errobs_205_full_errobs_362_validobs_362_fracobs_362_fluxobs_362_errobs_362_full_errobs_363_validobs_363_fracobs_363_fluxobs_363_errobs_363_full_errobs_364_validobs_364_fracobs_364_fluxobs_364_errobs_364_full_errobs_365_validobs_365_fracobs_365_fluxobs_365_errobs_365_full_errobs_366_validobs_366_fracobs_366_fluxobs_366_errobs_366_full_errobs_370_validobs_370_fracobs_370_fluxobs_370_errobs_370_full_errobs_371_validobs_371_fracobs_371_fluxobs_371_errobs_371_full_errobs_375_validobs_375_fracobs_375_fluxobs_375_errobs_375_full_errobs_376_validobs_376_fracobs_376_fluxobs_376_errobs_376_full_errobs_377_validobs_377_fracobs_377_fluxobs_377_errobs_377_full_errobs_379_validobs_379_fracobs_379_fluxobs_379_errobs_379_full_errobs_380_validobs_380_fracobs_380_fluxobs_380_errobs_380_full_errobs_381_validobs_381_fracobs_381_fluxobs_381_errobs_381_full_errobs_382_validobs_382_fracobs_382_fluxobs_382_errobs_382_full_errobs_383_validobs_383_fracobs_383_fluxobs_383_errobs_383_full_errobs_384_validobs_384_fracobs_384_fluxobs_384_errobs_384_full_errobs_385_validobs_385_fracobs_385_fluxobs_385_errobs_385_full_errobs_386_validobs_386_fracobs_386_fluxobs_386_errobs_386_full_errrest_120_validrest_120_fracrest_120_fluxrest_120_errrest_120_full_errrest_121_validrest_121_fracrest_121_fluxrest_121_errrest_121_full_errrest_218_validrest_218_fracrest_218_fluxrest_218_errrest_218_full_errrest_219_validrest_219_fracrest_219_fluxrest_219_errrest_219_full_errrest_270_validrest_270_fracrest_270_fluxrest_270_errrest_270_full_errrest_271_validrest_271_fracrest_271_fluxrest_271_errrest_271_full_errrest_272_validrest_272_fracrest_272_fluxrest_272_errrest_272_full_errrest_274_validrest_274_fracrest_274_fluxrest_274_errrest_274_full_errrest_153_validrest_153_fracrest_153_fluxrest_153_errrest_153_full_errrest_154_validrest_154_fracrest_154_fluxrest_154_errrest_154_full_errrest_155_validrest_155_fracrest_155_fluxrest_155_errrest_155_full_errrest_156_validrest_156_fracrest_156_fluxrest_156_errrest_156_full_errrest_157_validrest_157_fracrest_157_fluxrest_157_errrest_157_full_errrest_158_validrest_158_fracrest_158_fluxrest_158_errrest_158_full_errrest_159_validrest_159_fracrest_159_fluxrest_159_errrest_159_full_errrest_160_validrest_160_fracrest_160_fluxrest_160_errrest_160_full_errrest_161_validrest_161_fracrest_161_fluxrest_161_errrest_161_full_errrest_162_validrest_162_fracrest_162_fluxrest_162_errrest_162_full_errrest_163_validrest_163_fracrest_163_fluxrest_163_errrest_163_full_errrest_414_validrest_414_fracrest_414_fluxrest_414_errrest_414_full_errrest_415_validrest_415_fracrest_415_fluxrest_415_errrest_415_full_errrest_416_validrest_416_fracrest_416_fluxrest_416_errrest_416_full_errbetabeta_ref_fluxbeta_npixbeta_wlobeta_whibeta_nmaddla_npixdla_valuedla_uncbeta_cov_00beta_cov_01beta_cov_10beta_cov_11
str57int64float64float64str5str6float64int64str72str25int64int64int64int64str24int64int64float64float64float64float64float64float64float64float64float64float64float64float64int64int64int64int64float64float64float64float64float64float64str30float64float64int64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64str5int64float64str1float64float64float64float64float64str44int64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64float64int64float64str4str93float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64int64float64float64float64float64float64float64int64float64float64float64float64float64float64float64float64float64float64
excels-uds01-v4_g235m-f170lp_3543_109269.spec.fits10926934.35062413-5.14987821G235MF170LP6565.06jw03543001001_05101_00002_nrs1_f170lp_g235m_raw.113.3543_109269.fitsjw03543001001_02_msa.fits6121113excels-uds01-v4277511.60523814.57525732.81565710.66378920.186908350.1052767562.24394820.426148150.179134134.08941030.88038350.5585832650514341544340.056.9097370.00.0010.74323141737361758.03376650.9.4.dev7+g92b1aa639390.021056.3827754609.642638.1352113.53396.2305715.9514675----------------421.7884512.25353940.83299.549703--------77.9442211.366782-18.49447814.979722----10.2970218.836235160.588789.798167--------99.2030511.217545533.6721913.3871485.1325214.60536-8.73928312.896657----45.19025416.34102760.06665----17.633625------31.5436158.97263475.6516412.4923011478.719520.716312----------------------------------------------------------2.69166351.02926061.76236941.20841380.159368230.77346180.1092582641.5076850.097979590.941813950.096818451.17772080.1016513850.778270240.11489711.01618060.150892060.772057830.29294607-0.368061961.5045673-4700.3892830.57180.38631132.51916310.00.00.00.00.00.00.220102411.95523730.12842322.3575760.102184562.20457650.0804555341.8396890.072078581.95908140.0716446641.71625420.077689741.45217620.082387544.622439-----23.88004110.763638-4.21606810.945385-10.38003910.46370334.73071312.58505763.8607211.191996-11.49343810.9105421681.471822.707413-3.884336713.712671----92.76910413.244619303.342214.01827732.4074314.73695416.25057615.096581101.7147913.087306102.2770512.364596-1.530984611.467584-0.009823302-0.121039405----------------33.23996.5497913--------------224.4988918.628468----48.388504-15.68925----5.406689680.58066----43.5218914.8675652.3390548-4.0933743----30.003298341.56543------15.96317259.9941822.1586---------------------------------10.549018-1.8727355-4.63383915.34104628.01381-5.0520831117.2476-2.832585861.117584204.4885632.1279116.23694277.4847179.08182-1.00996474.01737501800.0------------------------------------------------------------True1724494.6228543G4.6215824.62285431.425.580.07257426primer-uds-north-grizli-v7.2-fix_phot.fits3978723.860.29176659943328620.00745835209426020350.3758223233385980.0074742586318044410.44621158445568650.0064404276493320010.57397989446778470.00551312306007662250.84356759235092180.0047415835913716241.00948827352212470.00471437536860440550.88930533721385170.0073197983988069980.93248024707031170.0064055659448006261.769778101145008432809883821.6441650.509837750.85152031.71898964.39068841693531123.12067562439001724.140659491718980.061100734.62244GBr--4.6224387590076680.74656688392377727750.00.0-1.0-1.027750.257349463347073260.10740292183922340.136369441103519170.1087719337910259827750.00013941477819138190.0-1.0-1.027757.565232436087342e-050.0-1.0-1.027753.422394938541863e-050.0-1.0-1.027750.183654826621415270.082098529788242750.167339419530450230.1334748319585305827750.99998559485371210.31932196326641320.0050462637331229960.00399020087175682427750.99999418430693290.25377922569408760.0073931426867225950.005859048616848037427750.99999978345780570.380335100717340950.0071669744241209750.00566243893578468727751.04458118603856030.51740153314432490.0053261519901072520.00412305361033587827750.95927286840844840.6745018550004720.0105662957176877980.00789007851842686627750.68647664612952060.57922019424957570.0202251259584320140.01465446647866136927750.99992020459972510.429097519374822430.0081240195359323830.00630806092881876227751.13398071359042230.395991407145957150.0091809659750243540.00697904475897215427750.91557593491247120.43441440496799260.0120061508619827320.00898695027133847127751.00037081021005840.92539139255793290.015876403163767970.01194562966864433427750.99996294443725140.60121304103825950.0209542296791421050.01528744591322381327750.9999523827689610.58475513434910330.0345968029849937140.02506850515324459527750.2717429488887240.3857865717627310.090411929514558330.0648628356886288427750.0008707652505335630.0-1.0-1.027750.00.0-1.0-1.027750.00423352556868361250.0-1.0-1.027750.00.0-1.0-1.027750.302974384383537470.07291371261998590.178755729743772570.142580822211009527750.00.0-1.0-1.027750.00.0-1.0-1.027750.00.0-1.0-1.027750.2211925775581921-0.105630516229403840.41362965779374120.329924593560386627751.00003370853336440.337386684000136660.0049501997133148840.00391512068109804227751.00011810691057980.45486659530942920.00437500333353669450.003411465472288634227751.0016996161901920.45049446997083050.0070855186844952330.00539840179414354427751.00000889894961120.325475925964423260.0054078435536040750.00427637038243981127751.00043913190193520.53305394955182110.0049041240905993880.003821276538650555327750.96756559461569060.67158570979165740.0105537958802706280.0079432480903028727750.95874680932730360.58336247608236220.0194153538647165760.01409035663350078327750.0106346762416460130.0-1.0-1.027750.00.0-1.0-1.027750.00.0-1.0-1.027750.00.0-1.0-1.027750.59465321859305130.143785609883504060.080656306506196570.0643317625525549327750.99917000009262020.436132376755631770.0075827461978066630.005882422201278385427750.99934847778788480.59722171450157180.0221660013538589050.016079926523872485----0--------------------
gto-wide-uds13-v4_g235h-f170lp_1215_1951.spec.fits195134.35065901-5.14988767G235HF170LP802.3894jw01215013001_04101_00001_nrs2_f170lp_g235h_raw.70.1215_1951.fitsjw01215013001_01_msa.fits2051170gto-wide-uds13-v4224311.662.56891422.0963542-0.4861431-0.76567411.25939890.493435320.82248261.58585581.53045342.4590352.1690035113672048400.0253.849960.0-0.0010.49176791738073094.40703850.9.5.dev2+g80b81b41604.7782913.674822432811.6072----------------------------139.0327984.556435------------------------153.8161261.984585--------4.036922578.6098345.68170580.60967-50.98967787.7149273.970177.34231--------853.96075----94.13828-------30.71053161.72949-------------------------------------------------------------------1.07939352.657897716.8558966.0944111.05317835.00594471.09056543.18874671.25973054.45998572.4125624-0.02466108812.6989640.00.00.00.00.00.00.00.00.00.03.1934314-15.3881870.00.00.00.00.00.04.4179719.6017792.19342238.9636851.65063677.1661411.33674758.3720351.21251074.26637741.23175954.64916231.15570223.88286881.08954664.6232686----47.930361.00501-43.6645273.04057-52.86496769.194349.7313574.62461-10.00257282.228905-74.39123564.71364----------------------------------------0.008058175-0.11264437------------------------------------27.334742--------------35.321674----0.901568310.652017-5.75122649.181498------154.39067-------7.458391------------------------------------10.362592-9.0206-10.35602611.604869-2.242879-16.51436------------------9.11737732200.0------------------------------------------------------------True1724494.6228543G4.6215824.62285431.855.580.07257426primer-uds-north-grizli-v7.2-fix_phot.fits3978723.860.29176659943328620.00745835209426020350.3758223233385980.0074742586318044410.44621158445568650.0064404276493320010.57397989446778470.00551312306007662250.84356759235092180.0047415835913716241.00948827352212470.00471437536860440550.88930533721385170.0073197983988069980.93248024707031170.0064055659448006261.769778101145008432809883821.6441650.509837750.85152031.71898964.39068841693531123.12067562439001724.140659491718980.061100734.62327AutoRedshift matches gto-wide-uds13-v3_g395h-f290lp_1215_1951 z=4.62004.6232685499098020.709456638881514622430.00.0-1.0-1.022430.083691590145044890.0-1.0-1.022430.000134752121311073870.0-1.0-1.022437.249783264163463e-050.0-1.0-1.022433.2292668859647866e-050.0-1.0-1.022430.0282562457731430070.0-1.0-1.022431.03306078109461890.7518296714297640.049730139772867070.03505414077391367422430.95240845068388980.62077772864057670.07365738401822490.05182535164970092422430.99999325117785880.89085270895061190.071847633338447870.0507420817880656822430.233106790855914180.81777576499431350.085669650468178290.0608689089796093322435.297289023649329e-060.0-1.0-1.022438.489538935327642e-050.0-1.0-1.022430.86066067963849080.81788035777383830.087375034796877010.06207465732495831622438.430932664970874e-060.0-1.0-1.022436.407891996424609e-060.0-1.0-1.022435.218707652222712e-060.0-1.0-1.022432.556004809596254e-060.0-1.0-1.022434.739559277072582e-060.0-1.0-1.022435.993161422598339e-050.0-1.0-1.022430.000148670000814431580.0-1.0-1.022430.00.0-1.0-1.022430.00135848498352280.0-1.0-1.022430.00.0-1.0-1.022430.0290870783938803680.0-1.0-1.022430.00.0-1.0-1.022430.00.0-1.0-1.022430.00.0-1.0-1.022430.00.0-1.0-1.022431.00465456271059830.79272402927996470.049087290061522040.0346340389621928322430.73806431842625480.85028849649057750.05570173491176360.03949338160150437522430.00.0-1.0-1.022431.00472001852256040.90226537416446920.053949736303127480.0380545011590011422430.3978842869223480.86323279992577970.059183548378377830.0420075875027193922430.00.0-1.0-1.022430.00.0-1.0-1.022430.00.0-1.0-1.022430.00.0-1.0-1.022430.00.0-1.0-1.022430.00.0-1.0-1.022430.319745270180893050.3072443703164660.087453013573204090.0613896897903287622430.68030230237934080.81331012677239940.090389079338019910.0642163815758274322430.00.0-1.0-1.0----0--------------------
gto-wide-uds13-v4_g395h-f290lp_1215_1951.spec.fits195134.35065901-5.14988767G395HF290LP875.3334jw01215013001_07101_00001_nrs1_f290lp_g395h_raw.70.1215_1951.fitsjw01215013001_01_msa.fits2071170gto-wide-uds13-v4218112.834.3332573.0323372-0.78633463-15.71353912.1728050.0238156060.373342618.1839220.8675502516.77023729.42786416131366436240.0253.84990.00.00099999980.49176791738073098.92303820.9.5.dev2+g80b81b41750.6662725.717321812691.675523.426165412.58319.938076307.09857--------------------------------1451.3488428.447899.84564273.67517----------------------------------------845.74225630.42847-------------------------------------------------------------------------------------14.73979216.29617312.9411331.02383393.397932-3.89114953.70977527.11492163.9599311-7.77667146.36292737.8243421.133274-1564.3295707.89160.00.00.00.00.00.00.00.020.1023549.972840.00.00.00.00.00.016.226713-5.6475356.898581519.8417325.6790743-17.2085844.401540812.0064473.8504024-5.29018263.576970310.4832843.5806997-0.0448386453.35363154.5170674-----------------------------6.600751413.0291735.67981398.38538----261.49884355.15863129.73485455.60483-77.98808304.731051188.4515320.5895-734.2784440.50098-461.14597403.66617-347.5903561.14636-0.05869624-0.35027233-----------------53.5106547.888203----------------------486.24625435.344------------------------213.96024-----------------------------------------------------------1.97491235.720995494.40948531.44778370.47738-1292.1211-123.25613-80.68548-96.065163.71737732200.0------------------------------------------------------------False1724494.6228543G4.6215824.62285431.855.580.07257426primer-uds-north-grizli-v7.2-fix_phot.fits3978723.860.29176659943328620.00745835209426020350.3758223233385980.0074742586318044410.44621158445568650.0064404276493320010.57397989446778470.00551312306007662250.84356759235092180.0047415835913716241.00948827352212470.00471437536860440550.88930533721385170.0073197983988069980.93248024707031170.0064055659448006261.769778101145008432809883821.6441650.509837750.85152031.71898964.39068841693531123.12067562439001724.140659491718980.0611007--------4.5170672327779310.564860788478507821810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.378372806033187350.30073378877350320.74446020128737310.490704770112601121810.99997725815630871.0849590560588270.53790136814100730.3258501599646858621810.46571906088239290.316456122429764450.90230311614666590.502937109961654621810.000103458983994215680.0-1.0-1.021810.97137510203228260.457418632290097940.68870623543304750.452032368299819321810.99997883684430160.63063891826112220.83783928458635390.521586851955781621810.99991777977921651.56958255509102430.73398784084696040.439270433698185521810.98992885070480440.33781143559150890.87274104008912750.488844949655945121810.72624105927876840.72878325700327311.62514510261262380.889636331857754621810.00020465581692088940.0-1.0-1.021818.095252757928491e-050.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.0254781012251645250.0-1.0-1.021810.82917427996905710.464369903428483330.5853868721160590.3825571880849472621810.00.0-1.0-1.021810.15580905088338265-0.122382824001332341.18902953049758420.79528513113441721811.0007955356758030.94723197765313770.59816004549351360.3701599690609964621810.7884540479681250.48609976891580190.79507711314552080.446923297492038321813.0335519041963066e-050.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.00.0-1.0-1.021810.86201012639976390.327003048833010730.88946548014562950.4967532214376828----0--------------------
gto-wide-uds13-v4_prism-clear_1215_1951.spec.fits195134.35065901-5.14988767PRISMCLEAR802.3893jw01215013001_03101_00002_nrs2_clear_prism_raw.70.1215_1951.fitsjw01215013001_01_msa.fits11170gto-wide-uds13-v446810.549125975.50183343.68473222.32431390.254639330.026172359.3024950.525888860.05672894815.8331740.813775360.2462121411359506280.0253.850.00.0010.49176791738073090.45903830.9.5.dev2+g80b81b42407.1675617.831468613.400448.81281322.620071-2.43898720.292538----------------560.3777546.5238871.5108562.164665-----209.0797133.749680.51485428.13365440.7857123.21867660.54590223.91554396.5061643.993908291.3935275.85304-716.07837280.8714657.39623118.88307514.75586138.25935124.9183669.4759116.87719589.80219-47.7029494.66547447.62188198.2818999.0163832.63919733.2995652.6719621.67851383.931694--202.1944241.93048-169.2497372.94585932.392551.2134742339.160470.73323---12.02763730.294546260.6562245.92269-66.2230432.629578--------------------------------241.5833727.795877126.8822133.298237228.2963342.90715840.98921224.23347313.5295441.53098260.0668413861.2147470.0654479861.2784380.068561481.10275630.0708167851.02494170.0630152150.890603240.066679060.863898460.071084390.6518140.082727430.892115350.0982066540.7999550.13572117.265528722.441030.55887010.202403320.8701770.26910640.256994630.492328142.70035514.0981360.9542901510.1803530.39419785.2171680.220239363.39105840.157952492.85738090.128474272.12954570.0949365351.66466440.068344154.621582-649.9793196.06087---------------------------------323.86383244.54553-----------------------------0.020892859-0.019805139164.59029193.0386363.44467219.52042--------43.720543-2.5788078--------17.805237-53.110912274.049278.2447525.7951---70.5157155.21557235.79257680.42899-28.52515644.31549115.00771-52.4203312.157452172.5845643.145094.735929-13.8351314.56473328.05231377.97484235.5395449.396652--18.168468-67.426506482.791441235.6241-14.254797377.87656-86.09008----------------195.16728151.96198334.706822837.588958.07976------------------------------48.91737732200.0------------------------------------------------------------True1724494.6228543G4.6215824.62285431.855.580.07257426primer-uds-north-grizli-v7.2-fix_phot.fits3978723.860.29176659943328620.00745835209426020350.3758223233385980.0074742586318044410.44621158445568650.0064404276493320010.57397989446778470.00551312306007662250.84356759235092180.0047415835913716241.00948827352212470.00471437536860440550.88930533721385170.0073197983988069980.93248024707031170.0064055659448006261.769778101145008432809883821.6441650.509837750.85152031.71898964.39068841693531123.12067562439001724.140659491718980.061100734.62158AutoRedshift matches gto-wide-uds13-v3_g395h-f290lp_1215_1951 z=4.62004.6215818223666230.97108918291320174680.99931187732236240.297345287368543040.0050725100880407350.0057671776762988734681.00499205547901840.393580777257697360.0067549192465975960.0075320346582501854680.99941215890904590.21051690707795250.0057406637807352140.0065002766154227234681.00483313872040240.31427745786778980.00569726255184620.0064755896321601454681.00578029542423250.36338551756438990.0055521402602133170.00632323308909434754680.99693787404460780.3900864612676870.0060319484282636830.0067404626344621284680.99955453896318470.474286999296680940.0060872528641253720.0066447087825195584680.99495833584639190.42216954786398170.0081933610346360960.0089540260979210974680.99663989150137570.53306877351952390.009209356172428410.0100744247050554624681.0001357431027050.65857271863069610.005403493880405120.0058763149914546994681.00013774560664180.73941494232403860.0063236972216453260.00657843647415761254681.0000135670620850.61663115284743460.0086959886832214480.0081208343963060634680.99739993435266640.51232150540810250.0095293157240517770.0101622162892088694680.99984093274026230.56265092498234650.0077732873586725840.0080419472496916694680.99953296666652480.53961413625242080.0081871839437407510.0082207090806280894680.99996587108445710.93435327855731410.0096785913263813280.0104726755672192384681.00001430394332270.61987059973792210.0105207143709352780.0100782244938471634680.99975442088574470.60463180932711350.0165839680306032330.0156686152139237964681.00044031382300380.61046818523429650.0206204812996903760.019065157769290854680.99975862164716910.61814436271827080.021076738530111410.0192613525788099644681.00056859887945460.30693433388927470.0055832518962781260.0063499448216635274681.00031046197706440.37617507728276210.0043129069829441250.0048818839758859834680.99041317083763460.318022419512767330.0060280626749118580.0068531235549381714681.01077843200428190.3896902934395620.0084034197504122640.0093420994158070054681.00652327683654640.296512597002468460.0069429862854918340.0079033900375131494680.98475649795752220.323206051845095540.0084701502530873330.0096144903341584164681.01571880805442260.359585871572219050.008705281222085330.0098802075696683044681.00886704295383730.39768709001436640.0106500060761624950.0118693498336187074680.99961701065201320.4906474263814740.0061741410105441290.0067498642919289564681.0001847524600220.56868094419133870.0052804875016632250.0056844996408233984681.00014711645800110.6178241197762920.0058993189105843390.006161023266307294681.00012791665653360.47702550757606390.00664227086801747350.007255405697753864680.99999698654922990.65482369503652180.0051249022700812130.0056172700972459554680.99999985151259550.75759543608605350.0068382874970659630.0072371383068888764680.99973554347417370.61947142879174430.0090745227272573030.0086168951713262984680.92147085716483560.6683456535239440.020388971554850260.018301673408882314680.00.0-1.0-1.04680.00.0-1.0-1.04680.00.0-1.0-1.04681.0271026874464680.39105166136900730.0079455096610144010.0087881830194002684680.99843097463695710.52304355412673430.008875569807922510.0094629194318379484681.00437793572066080.61768136609355050.0106988285627730890.010147776185419594-1.45652064793650850.3061252990650476440.1403621198618130.25671459195207271.19540543417518315.018.5631673665753035.1446091370229970.0021474377023419544-0.0001561310838130919-0.000156131083813091881.803142058546427e-05

#  see the description in the https://dawn-cph.github.io/dja/blog/2025/05/01/nirspec-merged-table-v4/
#  
RGB_URL = "https://grizli-cutout.herokuapp.com/thumb?size=1.5&scl=2.0&asinh=True&filters=f115w-clear%2Cf277w-clear%2Cf444w-clear&rgb_scl=1.5%2C0.74%2C1.3&pl=2&coord={ra}%2C{dec}"
msa['metafile'] = [m.split('_')[0] for m in msa['msamet']]
SLIT_URL = "https://grizli-cutout.herokuapp.com/thumb?size=1.5&scl=4.0&invert=True&filters=f444w-clear&rgb_scl=1.5%2C0.74%2C1.3&pl=2&coord={ra}%2C{dec}&nirspec=True&dpi_scale=6&nrs_lw=0.5&nrs_alpha=0.8&metafile={metafile}"
FITS_URL = "https://s3.amazonaws.com/msaexp-nirspec/extractions/{root}/{file}"

msa['Thumb'] = [
    "<img src=\"{0}\" height=200px>".format(
        RGB_URL.format(**row['ra','dec'])
    )
    for row in msa
]

msa['Slit_Thumb'] = [
    "<img src=\"{0}\" height=200px>".format(
        SLIT_URL.format(**row['ra','dec','metafile'])
    )
    for row in msa
]

msa['Spectrum_fnu'] = [
    "<img src=\"{0}\" height=200px>".format(
        FITS_URL.format(**row['root','file']).replace('.spec.fits', '.fnu.png')
    )
    for row in msa
]

msa['Spectrum_flam'] = [
    "<img src=\"{0}\" height=200px>".format(
        FITS_URL.format(**row['root','file']).replace('.spec.fits', '.flam.png')
    )
    for row in msa
]


df = msa['root','file','z_best','phot_mass','eqw_ha_nii','Thumb','Slit_Thumb','Spectrum_fnu', 'Spectrum_flam'].to_pandas()

display(Markdown(df.to_markdown()))
  root file z_best phot_mass eqw_ha_nii Thumb Slit_Thumb Spectrum_fnu Spectrum_flam
0 excels-uds01-v4 excels-uds01-v4_g235m-f170lp_3543_109269.spec.fits 4.62285 3.28099e+10 nan
1 gto-wide-uds13-v4 gto-wide-uds13-v4_g235h-f170lp_1215_1951.spec.fits 4.62285 3.28099e+10 nan
2 gto-wide-uds13-v4 gto-wide-uds13-v4_g395h-f290lp_1215_1951.spec.fits 4.62285 3.28099e+10 nan
3 gto-wide-uds13-v4 gto-wide-uds13-v4_prism-clear_1215_1951.spec.fits 4.62285 3.28099e+10 2274.05 
# Reading the spectrum

for i in range(len(df)):
    spec_file = df['file'][i]
    row = msa[msa['file'] == spec_file][0]
    spec = msaexp.spectrum.SpectrumSampler(FITS_URL.format(**row))

    ## 
    con_mask = spec['full_err']==0 
    con_mask |= spec['full_err']> spec['flux']

    galaxy_masked = np.ma.masked_where(con_mask, spec['flux'])
    plt.plot(spec['wave'], galaxy_masked,
         label="{file}\nz={z_best:.3f}".format(**row), alpha=0.5)


plt.xlim([0.8,6])
plt.ylim(top=galaxy_masked.max()*2)
plt.semilogx()
plt.legend(fontsize=7, ncol=1)

<matplotlib.legend.Legend at 0x3117895e0>

png

]]>Minju LeeAPI Documentation Posted2025-11-07T21:36:13+00:002025-11-07T21:36:13+00:00https://dawn-cph.github.io/dja/blog/2025/11/07/api-documentation-posted news documentation

Comprehensive documentation on the API functionality at https://grizli-cutout.herokuapp.com/ is now available at https://dawn-cph.github.io/dja/general/api_summary/.

Image thumbnails

]]>Gabriel BrammerDJA+Grizli Cutout WFSS spectra2025-05-16T20:59:38+00:002025-05-16T20:59:38+00:00https://dawn-cph.github.io/dja/blog/2025/05/16/simplified_cutout_wfss spectroscopy wfss grism demo

DJA+Grizli Cutout WFSS spectra

Simplified extraction of the grism spectrum for an arbitrary sky position within some WFSS exposure processed with DJA/Grizli.

Totally ignore the full-field contamination model and fit the 2D spectra with flexible spline components to model out continuum / contamination and isolate emission lines. No detection image, catalog, segmentation image required!.

2025-05-16-simplified_cutout_wfss.ipynb Notebook

import os
os.environ['CRDS_CONTEXT'] = "jwst_1322.pmap"
os.environ['NIRCAM_CONF_VERSION'] = "V9"

Build dependencies

E.g., on Google Colab.

Restart runtime if dependencies installed.

try:
    import grizli
except ImportError:
    # Install dependencies - restart runtime if performed
    ! pip install grizli[aws,jwst] msaexp
    ! pip install git+https://github.com/karllark/dust_attenuation.git

Environment variables

Requires an AWS account for downloading exposures, though the functionality shouldn’t generate any charges. If running in Google Colab, set the AWS credentials variables in the “Secrets” tab at the left with the key icon.

import os

env = {
    'CRDS_PATH': '/tmp/crds_cache',
    'CRDS_SERVER_URL': 'https://jwst-crds.stsci.edu'
}
for k in env:
    if os.getenv(k) is None:
        print(f'set {k} = {env[k]}')
        os.environ[k] = env[k]

try:
    from google.colab import userdata
    for k in ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY']:
        os.environ[k] = userdata.get(k)
except ImportError:
    pass

if os.getenv('AWS_ACCESS_KEY_ID') is None:
    print('AWS credentials not found.  Set them in the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables')

set CRDS_PATH = /tmp/crds_cache
set CRDS_SERVER_URL = https://jwst-crds.stsci.edu

Download config files

from grizli import utils
from grizli.aws import db

CONF_DIR = os.path.join(utils.GRIZLI_PATH, "CONF")
if not os.path.exists(CONF_DIR):
    os.makedirs(CONF_DIR)

utils.fetch_config_files(get_sky=False, get_wfc3=False, get_jwst=True)

if not os.path.exists("fsps_line_templ.fits"):
    db.download_s3_file("s3://grizli-v2/junk/fsps_line_templ.fits")

Set ROOT_PATH=/content
Config directory: /usr/local/lib/python3.11/dist-packages/grizli/data//CONF
Get jwst-grism-conf.tar.gz
Get niriss.conf.220725.tar.gz
Get WFC3IR_extended_PSF.v1.tar.gz
Get PSFSTD_WFC3IR_F105W.fits


WARNING: VerifyWarning: Verification reported errors: [astropy.io.fits.verify]
WARNING:astropy:VerifyWarning: Verification reported errors:
WARNING: VerifyWarning: HDU 0: [astropy.io.fits.verify]
WARNING:astropy:VerifyWarning: HDU 0:
WARNING: VerifyWarning:     Card 10: [astropy.io.fits.verify]
WARNING:astropy:VerifyWarning:     Card 10:
WARNING: VerifyWarning:         Card keyword 'NXPSFs' is not upper case.  Fixed 'NXPSFS' card to meet the FITS standard. [astropy.io.fits.verify]
WARNING:astropy:VerifyWarning:         Card keyword 'NXPSFs' is not upper case.  Fixed 'NXPSFS' card to meet the FITS standard.
WARNING: VerifyWarning:     Card 11: [astropy.io.fits.verify]
WARNING:astropy:VerifyWarning:     Card 11:
WARNING: VerifyWarning:         Card keyword 'NYPSFs' is not upper case.  Fixed 'NYPSFS' card to meet the FITS standard. [astropy.io.fits.verify]
WARNING:astropy:VerifyWarning:         Card keyword 'NYPSFs' is not upper case.  Fixed 'NYPSFS' card to meet the FITS standard.
WARNING: VerifyWarning: Note: astropy.io.fits uses zero-based indexing.
 [astropy.io.fits.verify]
WARNING:astropy:VerifyWarning: Note: astropy.io.fits uses zero-based indexing.



Get PSFSTD_WFC3IR_F125W.fits
Get PSFSTD_WFC3IR_F140W.fits
Get PSFSTD_WFC3IR_F160W.fits
Get PSFSTD_WFC3IR_F110W.fits
Get PSFSTD_WFC3IR_F127M.fits


WARNING: VerifyWarning:     Card 12: [astropy.io.fits.verify]
WARNING:astropy:VerifyWarning:     Card 12:


Templates directory: /usr/local/lib/python3.11/dist-packages/grizli/data//templates
Get stars_pickles.npy
Get stars_bpgs.npy
ln -s stars_pickles.npy stars.npy
s3://grizli-v2/junk/fsps_line_templ.fits > ./fsps_line_templ.fits

# GRISM_NIRCAM (N. Pirzkal) for NIRCam
if 1:
    prev = os.getcwd()

    os.chdir(os.path.join(CONF_DIR, 'GRISM_NIRCAM'))

    if not os.path.exists('V9'):
        ! git clone https://github.com/npirzkal/GRISM_NIRCAM.git GRISM_NIRCAM_repo
        ! ln -s GRISM_NIRCAM_repo/* ./
    else:
        print('GRISM_NIRCAM files found')

    os.chdir(prev)

Cloning into 'GRISM_NIRCAM_repo'...
remote: Enumerating objects: 1491, done.
remote: Counting objects: 100% (353/353), done.
remote: Compressing objects: 100% (194/194), done.
remote: Total 1491 (delta 159), reused 353 (delta 159), pack-reused 1138 (from 1)
Receiving objects: 100% (1491/1491), 219.22 MiB | 2.69 MiB/s, done.
Resolving deltas: 100% (710/710), done.
Updating files: 100% (986/986), done.

from grizli import utils, grismconf
grismconf.download_jwst_crds_references()

Use NIRCAM_CONF_VERSION = V9
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_system_datalvl_0002.rmap      694 bytes  (1 / 202 files) (0 / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_system_calver_0046.rmap    5.2 K bytes  (2 / 202 files) (694 / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_system_0045.imap          385 bytes  (3 / 202 files) (5.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_wavelengthrange_0024.rmap    1.4 K bytes  (4 / 202 files) (6.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_wavecorr_0005.rmap      884 bytes  (5 / 202 files) (7.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_superbias_0074.rmap   33.8 K bytes  (6 / 202 files) (8.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_sflat_0026.rmap   20.6 K bytes  (7 / 202 files) (42.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_saturation_0018.rmap    2.0 K bytes  (8 / 202 files) (62.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_refpix_0015.rmap    1.6 K bytes  (9 / 202 files) (64.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_readnoise_0025.rmap    2.6 K bytes  (10 / 202 files) (66.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_photom_0013.rmap      958 bytes  (11 / 202 files) (69.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pathloss_0008.rmap    1.2 K bytes  (12 / 202 files) (70.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-whitelightstep_0001.rmap      777 bytes  (13 / 202 files) (71.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-spec2pipeline_0013.rmap    2.1 K bytes  (14 / 202 files) (72.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-resamplespecstep_0002.rmap      709 bytes  (15 / 202 files) (74.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-outlierdetectionstep_0005.rmap    1.1 K bytes  (16 / 202 files) (74.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-jumpstep_0005.rmap      810 bytes  (17 / 202 files) (76.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-image2pipeline_0008.rmap    1.0 K bytes  (18 / 202 files) (76.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-detector1pipeline_0003.rmap    1.1 K bytes  (19 / 202 files) (77.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-darkpipeline_0003.rmap      872 bytes  (20 / 202 files) (78.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_pars-darkcurrentstep_0001.rmap      622 bytes  (21 / 202 files) (79.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_ote_0030.rmap    1.3 K bytes  (22 / 202 files) (80.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_msaoper_0016.rmap    1.5 K bytes  (23 / 202 files) (81.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_msa_0027.rmap    1.3 K bytes  (24 / 202 files) (83.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_mask_0039.rmap    2.7 K bytes  (25 / 202 files) (84.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_linearity_0017.rmap    1.6 K bytes  (26 / 202 files) (87.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_ipc_0006.rmap      876 bytes  (27 / 202 files) (88.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_ifuslicer_0017.rmap    1.5 K bytes  (28 / 202 files) (89.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_ifupost_0019.rmap    1.5 K bytes  (29 / 202 files) (91.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_ifufore_0017.rmap    1.5 K bytes  (30 / 202 files) (92.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_gain_0023.rmap    1.8 K bytes  (31 / 202 files) (94.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_fpa_0028.rmap    1.3 K bytes  (32 / 202 files) (95.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_fore_0026.rmap    5.0 K bytes  (33 / 202 files) (97.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_flat_0015.rmap    3.8 K bytes  (34 / 202 files) (102.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_fflat_0026.rmap    7.2 K bytes  (35 / 202 files) (105.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_extract1d_0018.rmap    2.3 K bytes  (36 / 202 files) (113.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_disperser_0028.rmap    5.7 K bytes  (37 / 202 files) (115.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_dflat_0007.rmap    1.1 K bytes  (38 / 202 files) (121.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_dark_0069.rmap   32.6 K bytes  (39 / 202 files) (122.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_cubepar_0015.rmap      966 bytes  (40 / 202 files) (154.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_collimator_0026.rmap    1.3 K bytes  (41 / 202 files) (155.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_camera_0026.rmap    1.3 K bytes  (42 / 202 files) (157.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_barshadow_0007.rmap    1.8 K bytes  (43 / 202 files) (158.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_area_0018.rmap    6.3 K bytes  (44 / 202 files) (160.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_apcorr_0009.rmap    5.6 K bytes  (45 / 202 files) (166.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nirspec_0387.imap       5.7 K bytes  (46 / 202 files) (171.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_wfssbkg_0008.rmap    3.1 K bytes  (47 / 202 files) (177.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_wavemap_0008.rmap    2.2 K bytes  (48 / 202 files) (180.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_wavelengthrange_0006.rmap      862 bytes  (49 / 202 files) (183.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_trappars_0004.rmap      753 bytes  (50 / 202 files) (183.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_trapdensity_0005.rmap      705 bytes  (51 / 202 files) (184.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_throughput_0005.rmap    1.3 K bytes  (52 / 202 files) (185.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_superbias_0030.rmap    7.4 K bytes  (53 / 202 files) (186.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_specwcs_0014.rmap    3.1 K bytes  (54 / 202 files) (194.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_spectrace_0008.rmap    2.3 K bytes  (55 / 202 files) (197.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_specprofile_0008.rmap    2.4 K bytes  (56 / 202 files) (199.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_speckernel_0006.rmap    1.0 K bytes  (57 / 202 files) (201.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_saturation_0015.rmap      829 bytes  (58 / 202 files) (202.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_readnoise_0011.rmap      987 bytes  (59 / 202 files) (203.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_photom_0036.rmap    1.3 K bytes  (60 / 202 files) (204.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_persat_0007.rmap      674 bytes  (61 / 202 files) (205.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pathloss_0003.rmap      758 bytes  (62 / 202 files) (206.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pastasoss_0004.rmap      818 bytes  (63 / 202 files) (207.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-undersamplecorrectionstep_0001.rmap      904 bytes  (64 / 202 files) (208.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-tweakregstep_0012.rmap    3.1 K bytes  (65 / 202 files) (209.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-spec2pipeline_0008.rmap      984 bytes  (66 / 202 files) (212.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-sourcecatalogstep_0002.rmap    2.3 K bytes  (67 / 202 files) (213.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-resamplestep_0002.rmap      687 bytes  (68 / 202 files) (215.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-outlierdetectionstep_0004.rmap    2.7 K bytes  (69 / 202 files) (216.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-jumpstep_0007.rmap    6.4 K bytes  (70 / 202 files) (218.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-image2pipeline_0005.rmap    1.0 K bytes  (71 / 202 files) (225.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-detector1pipeline_0002.rmap    1.0 K bytes  (72 / 202 files) (226.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-darkpipeline_0002.rmap      868 bytes  (73 / 202 files) (227.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-darkcurrentstep_0001.rmap      591 bytes  (74 / 202 files) (228.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_pars-chargemigrationstep_0004.rmap    5.7 K bytes  (75 / 202 files) (228.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_nrm_0005.rmap      663 bytes  (76 / 202 files) (234.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_mask_0022.rmap    1.3 K bytes  (77 / 202 files) (235.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_linearity_0022.rmap      961 bytes  (78 / 202 files) (236.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_ipc_0007.rmap      651 bytes  (79 / 202 files) (237.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_gain_0011.rmap      797 bytes  (80 / 202 files) (238.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_flat_0023.rmap    5.9 K bytes  (81 / 202 files) (238.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_filteroffset_0010.rmap      853 bytes  (82 / 202 files) (244.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_extract1d_0007.rmap      905 bytes  (83 / 202 files) (245.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_drizpars_0004.rmap      519 bytes  (84 / 202 files) (246.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_distortion_0025.rmap    3.4 K bytes  (85 / 202 files) (246.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_dark_0034.rmap    7.5 K bytes  (86 / 202 files) (250.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_area_0014.rmap    2.7 K bytes  (87 / 202 files) (257.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_apcorr_0010.rmap    4.3 K bytes  (88 / 202 files) (260.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_abvegaoffset_0004.rmap    1.4 K bytes  (89 / 202 files) (264.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_niriss_0267.imap        5.8 K bytes  (90 / 202 files) (266.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_wfssbkg_0004.rmap    7.2 K bytes  (91 / 202 files) (272.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_wavelengthrange_0010.rmap      996 bytes  (92 / 202 files) (279.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_tsophot_0003.rmap      896 bytes  (93 / 202 files) (280.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_trappars_0003.rmap    1.6 K bytes  (94 / 202 files) (281.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_trapdensity_0003.rmap    1.6 K bytes  (95 / 202 files) (282.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_superbias_0018.rmap   16.2 K bytes  (96 / 202 files) (284.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_specwcs_0022.rmap    7.1 K bytes  (97 / 202 files) (300.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_sirskernel_0002.rmap      671 bytes  (98 / 202 files) (307.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_saturation_0010.rmap    2.2 K bytes  (99 / 202 files) (308.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_readnoise_0025.rmap   23.2 K bytes  (100 / 202 files) (310.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_psfmask_0008.rmap   28.4 K bytes  (101 / 202 files) (333.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_photom_0028.rmap    3.4 K bytes  (102 / 202 files) (362.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_persat_0005.rmap    1.6 K bytes  (103 / 202 files) (365.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-whitelightstep_0003.rmap    1.5 K bytes  (104 / 202 files) (367.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-tweakregstep_0003.rmap    4.5 K bytes  (105 / 202 files) (368.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-spec2pipeline_0008.rmap      984 bytes  (106 / 202 files) (372.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-sourcecatalogstep_0002.rmap    4.6 K bytes  (107 / 202 files) (373.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-resamplestep_0002.rmap      687 bytes  (108 / 202 files) (378.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-outlierdetectionstep_0003.rmap      940 bytes  (109 / 202 files) (379.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-jumpstep_0005.rmap      806 bytes  (110 / 202 files) (380.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-image2pipeline_0003.rmap    1.0 K bytes  (111 / 202 files) (381.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-detector1pipeline_0003.rmap    1.0 K bytes  (112 / 202 files) (382.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-darkpipeline_0002.rmap      868 bytes  (113 / 202 files) (383.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_pars-darkcurrentstep_0001.rmap      618 bytes  (114 / 202 files) (383.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_mask_0011.rmap    3.5 K bytes  (115 / 202 files) (384.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_linearity_0011.rmap    2.4 K bytes  (116 / 202 files) (388.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_ipc_0003.rmap    2.0 K bytes  (117 / 202 files) (390.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_gain_0016.rmap    2.1 K bytes  (118 / 202 files) (392.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_flat_0027.rmap   51.7 K bytes  (119 / 202 files) (394.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_filteroffset_0004.rmap    1.4 K bytes  (120 / 202 files) (446.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_extract1d_0004.rmap      842 bytes  (121 / 202 files) (447.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_drizpars_0001.rmap      519 bytes  (122 / 202 files) (448.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_distortion_0033.rmap   53.4 K bytes  (123 / 202 files) (449.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_dark_0046.rmap   26.4 K bytes  (124 / 202 files) (502.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_area_0012.rmap   33.5 K bytes  (125 / 202 files) (528.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_apcorr_0008.rmap    4.3 K bytes  (126 / 202 files) (562.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_abvegaoffset_0003.rmap    1.3 K bytes  (127 / 202 files) (566.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_nircam_0301.imap        5.6 K bytes  (128 / 202 files) (567.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_wavelengthrange_0027.rmap      929 bytes  (129 / 202 files) (573.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_tsophot_0004.rmap      882 bytes  (130 / 202 files) (574.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_straymask_0009.rmap      987 bytes  (131 / 202 files) (575.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_specwcs_0042.rmap    5.8 K bytes  (132 / 202 files) (576.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_saturation_0015.rmap    1.2 K bytes  (133 / 202 files) (582.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_rscd_0008.rmap     1.0 K bytes  (134 / 202 files) (583.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_resol_0006.rmap      790 bytes  (135 / 202 files) (584.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_reset_0026.rmap    3.9 K bytes  (136 / 202 files) (585.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_regions_0033.rmap    5.2 K bytes  (137 / 202 files) (588.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_readnoise_0023.rmap    1.6 K bytes  (138 / 202 files) (594.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_psfmask_0009.rmap    2.1 K bytes  (139 / 202 files) (595.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_psf_0002.rmap        753 bytes  (140 / 202 files) (597.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_photom_0056.rmap    3.7 K bytes  (141 / 202 files) (598.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pathloss_0005.rmap      866 bytes  (142 / 202 files) (602.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-whitelightstep_0003.rmap      912 bytes  (143 / 202 files) (603.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-tweakregstep_0003.rmap    1.8 K bytes  (144 / 202 files) (604.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-spec3pipeline_0009.rmap      816 bytes  (145 / 202 files) (605.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-spec2pipeline_0012.rmap    1.3 K bytes  (146 / 202 files) (606.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-sourcecatalogstep_0003.rmap    1.9 K bytes  (147 / 202 files) (608.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-resamplestep_0002.rmap      677 bytes  (148 / 202 files) (610.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-resamplespecstep_0002.rmap      706 bytes  (149 / 202 files) (610.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-outlierdetectionstep_0017.rmap    3.4 K bytes  (150 / 202 files) (611.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-jumpstep_0011.rmap    1.6 K bytes  (151 / 202 files) (614.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-image2pipeline_0007.rmap      983 bytes  (152 / 202 files) (616.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-extract1dstep_0003.rmap      807 bytes  (153 / 202 files) (617.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-emicorrstep_0003.rmap      796 bytes  (154 / 202 files) (618.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-detector1pipeline_0010.rmap    1.6 K bytes  (155 / 202 files) (618.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-darkpipeline_0002.rmap      860 bytes  (156 / 202 files) (620.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_pars-darkcurrentstep_0002.rmap      683 bytes  (157 / 202 files) (621.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_mrsxartcorr_0002.rmap    2.2 K bytes  (158 / 202 files) (622.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_mrsptcorr_0005.rmap    2.0 K bytes  (159 / 202 files) (624.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_mask_0023.rmap     3.5 K bytes  (160 / 202 files) (626.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_linearity_0018.rmap    2.8 K bytes  (161 / 202 files) (629.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_ipc_0008.rmap        700 bytes  (162 / 202 files) (632.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_gain_0013.rmap     3.9 K bytes  (163 / 202 files) (633.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_fringefreq_0003.rmap    1.4 K bytes  (164 / 202 files) (637.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_fringe_0019.rmap    3.9 K bytes  (165 / 202 files) (638.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_flat_0065.rmap    15.5 K bytes  (166 / 202 files) (642.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_filteroffset_0025.rmap    2.5 K bytes  (167 / 202 files) (657.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_extract1d_0020.rmap    1.4 K bytes  (168 / 202 files) (660.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_emicorr_0003.rmap      663 bytes  (169 / 202 files) (661.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_drizpars_0002.rmap      511 bytes  (170 / 202 files) (662.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_distortion_0040.rmap    4.9 K bytes  (171 / 202 files) (662.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_dark_0036.rmap     4.4 K bytes  (172 / 202 files) (667.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_cubepar_0017.rmap      800 bytes  (173 / 202 files) (672.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_area_0015.rmap       866 bytes  (174 / 202 files) (673.0 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_apcorr_0019.rmap    5.0 K bytes  (175 / 202 files) (673.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_abvegaoffset_0003.rmap    1.3 K bytes  (176 / 202 files) (678.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_miri_0423.imap          5.8 K bytes  (177 / 202 files) (680.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_trappars_0004.rmap      903 bytes  (178 / 202 files) (685.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_trapdensity_0006.rmap      930 bytes  (179 / 202 files) (686.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_superbias_0017.rmap    3.8 K bytes  (180 / 202 files) (687.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_saturation_0009.rmap      779 bytes  (181 / 202 files) (691.5 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_readnoise_0011.rmap    1.3 K bytes  (182 / 202 files) (692.3 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_photom_0014.rmap    1.1 K bytes  (183 / 202 files) (693.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_persat_0006.rmap      884 bytes  (184 / 202 files) (694.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_pars-tweakregstep_0002.rmap      850 bytes  (185 / 202 files) (695.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_pars-sourcecatalogstep_0001.rmap      636 bytes  (186 / 202 files) (696.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_pars-outlierdetectionstep_0001.rmap      654 bytes  (187 / 202 files) (697.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_pars-image2pipeline_0005.rmap      974 bytes  (188 / 202 files) (697.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_pars-detector1pipeline_0002.rmap    1.0 K bytes  (189 / 202 files) (698.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_pars-darkpipeline_0002.rmap      856 bytes  (190 / 202 files) (699.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_mask_0023.rmap      1.1 K bytes  (191 / 202 files) (700.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_linearity_0015.rmap      925 bytes  (192 / 202 files) (701.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_ipc_0003.rmap         614 bytes  (193 / 202 files) (702.6 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_gain_0010.rmap        890 bytes  (194 / 202 files) (703.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_flat_0009.rmap      1.1 K bytes  (195 / 202 files) (704.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_distortion_0011.rmap    1.2 K bytes  (196 / 202 files) (705.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_dark_0017.rmap      4.3 K bytes  (197 / 202 files) (706.4 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_area_0010.rmap      1.2 K bytes  (198 / 202 files) (710.7 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_apcorr_0004.rmap    4.0 K bytes  (199 / 202 files) (711.9 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_abvegaoffset_0002.rmap    1.3 K bytes  (200 / 202 files) (715.8 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_fgs_0118.imap           5.1 K bytes  (201 / 202 files) (717.1 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/mappings/jwst/jwst_1322.pmap                 580 bytes  (202 / 202 files) (722.2 K / 722.8 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_photom_0162.fits    1.7 M bytes  (1 / 2 files) (0 / 1.7 M bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0186.asdf    9.3 K bytes  (2 / 2 files) (1.7 M / 1.7 M bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_photom_0163.fits    1.7 M bytes  (1 / 2 files) (0 / 1.7 M bytes)


crds_reffiles: NIRCAM F277W GRISMR A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0186.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0160.asdf    9.3 K bytes  (2 / 2 files) (1.7 M / 1.7 M bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0171.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F277W GRISMR B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0160.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0174.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F277W GRISMC A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0171.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0184.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F277W GRISMC B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0174.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0199.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F356W GRISMR A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0184.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0165.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F356W GRISMR B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0199.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0182.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F356W GRISMC A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0165.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0178.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F356W GRISMC B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0182.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0164.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F410M GRISMR A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0178.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0181.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F410M GRISMR B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0164.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap
crds_reffiles: NIRCAM F410M GRISMC A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0181.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0159.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0190.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F410M GRISMC B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0159.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap
crds_reffiles: NIRCAM F444W GRISMR A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0190.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0187.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0173.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F444W GRISMR B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0187.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0203.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F444W GRISMC A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0173.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0191.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F444W GRISMC B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0203.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0179.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F460M GRISMR A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0191.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0195.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F460M GRISMR B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0179.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0169.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F460M GRISMC A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0195.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0192.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F460M GRISMC B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0169.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0197.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F480M GRISMR A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0192.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0180.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F480M GRISMR B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0197.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/nircam/jwst_nircam_specwcs_0168.asdf    9.3 K bytes  (1 / 1 files) (0 / 9.3 K bytes)


crds_reffiles: NIRCAM F480M GRISMC A (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0162.fits jwst_nircam_specwcs_0180.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap
crds_reffiles: NIRCAM F480M GRISMC B (jwst_1322.pmap)
crds_reffiles: jwst_nircam_photom_0163.fits jwst_nircam_specwcs_0168.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_photom_0041.fits    3.6 M bytes  (1 / 2 files) (0 / 3.6 M bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0050.asdf   20.6 K bytes  (2 / 2 files) (3.6 M / 3.6 M bytes)
CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0047.asdf   20.6 K bytes  (1 / 1 files) (0 / 20.6 K bytes)


crds_reffiles: NIRISS GR150R F090W A (jwst_1322.pmap)
crds_reffiles: jwst_niriss_photom_0041.fits jwst_niriss_specwcs_0050.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf   24.4 K bytes  (1 / 1 files) (0 / 24.4 K bytes)


crds_reffiles: NIRISS GR150C F090W A (jwst_1322.pmap)
crds_reffiles: jwst_niriss_photom_0041.fits jwst_niriss_specwcs_0047.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0053.asdf   27.4 K bytes  (1 / 1 files) (0 / 27.4 K bytes)


crds_reffiles: NIRISS GR150R F115W A (jwst_1322.pmap)
crds_reffiles: jwst_niriss_photom_0041.fits jwst_niriss_specwcs_0054.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf   27.5 K bytes  (1 / 1 files) (0 / 27.5 K bytes)


crds_reffiles: NIRISS GR150C F115W A (jwst_1322.pmap)
crds_reffiles: jwst_niriss_photom_0041.fits jwst_niriss_specwcs_0053.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0055.asdf   30.5 K bytes  (1 / 1 files) (0 / 30.5 K bytes)


crds_reffiles: NIRISS GR150R F150W A (jwst_1322.pmap)
crds_reffiles: jwst_niriss_photom_0041.fits jwst_niriss_specwcs_0056.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf   27.5 K bytes  (1 / 1 files) (0 / 27.5 K bytes)


crds_reffiles: NIRISS GR150C F150W A (jwst_1322.pmap)
crds_reffiles: jwst_niriss_photom_0041.fits jwst_niriss_specwcs_0055.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap


CRDS - INFO -  Fetching  /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0051.asdf   30.5 K bytes  (1 / 1 files) (0 / 30.5 K bytes)


crds_reffiles: NIRISS GR150R F200W A (jwst_1322.pmap)
crds_reffiles: jwst_niriss_photom_0041.fits jwst_niriss_specwcs_0052.asdf
ENV CRDS_CONTEXT = jwst_1322.pmap
crds_reffiles: NIRISS GR150C F200W A (jwst_1322.pmap)
crds_reffiles: jwst_niriss_photom_0041.fits jwst_niriss_specwcs_0051.asdf

Imports

Can skip to here if config files and dependencies already installed above.

import os

import numpy as np
import matplotlib.pyplot as plt

import astropy.io.fits as pyfits
import astropy.wcs as pywcs

import grizli.jwst_utils
grizli.jwst_utils.set_quiet_logging()

from grizli.aws import db
from grizli import utils, grismconf
from grizli.aws import sky_wfss

utils.set_warnings()

import grizli
import msaexp

from IPython.display import Image

print('grizli version: ', grizli.__version__)
print('msaexp version: ', msaexp.__version__)


grizli version:  1.12.14
msaexp version:  0.9.8

from importlib import reload
reload(sky_wfss)

prefix = 'dja-grism'
kwargs = {}

# Segmentation image for NIRISS demo
if not os.path.exists('gds-sw-grizli-v7.0-ir_seg.fits'):
    ! wget https://s3.amazonaws.com/grizli-v2/JwstMosaics/v7/gds-sw-grizli-v7.0-ir_seg.fits.gz
    ! gunzip gds-sw-grizli-v7.0-ir_seg.fits.gz

--2025-05-16 19:27:21--  https://s3.amazonaws.com/grizli-v2/JwstMosaics/v7/gds-sw-grizli-v7.0-ir_seg.fits.gz
Resolving s3.amazonaws.com (s3.amazonaws.com)... 3.5.9.70, 54.231.192.120, 16.182.105.32, ...
Connecting to s3.amazonaws.com (s3.amazonaws.com)|3.5.9.70|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5996324 (5.7M) [binary/octet-stream]
Saving to: ‘gds-sw-grizli-v7.0-ir_seg.fits.gz’

gds-sw-grizli-v7.0- 100%[===================>]   5.72M  14.4MB/s    in 0.4s    

2025-05-16 19:27:22 (14.4 MB/s) - ‘gds-sw-grizli-v7.0-ir_seg.fits.gz’ saved [5996324/5996324]

Set source and extract spectrum

All you really need to specify is a set of target coordinates (ra, dec) and a list of the desired grism names to search for.

The script

  1. Queries and downloads all DJA-processed exposures whose corner footprint contains the requested point. Note that there might not be a usable spectrum in a particular exposure if the nominal coordinate is within the WCS “footprint” of the grism exposure but the dispersed spectrum falls off of the detector.
  2. Pulls a FITS cutout from the DJA/grizli API to use for the direct image. No direct image is needed for the basic extraction and redshift fitting, which here uses a simplified Gaussian model for the optimal extractions, but the direct image is needed for the full grizli functionality (line maps, etc.)
  3. Generates a grizli.multifit.MultiBeam object of 2D cutouts from the grism exposures.

NIRISS

If a segmentation_image is provided, the source ID will be read from the specified target coordinates and the segmentation image will be used for a crude zeroth-order mask.

reload(sky_wfss)

## NIRISS test
ra, dec = 53.1666323, -27.8690371 # multiple lines

# segmentation_image = "gds-sw-grizli-v7.0-ir_seg.fits"
segmentation_image = None

mb = sky_wfss.extract_from_coords(
    ra=ra, dec=dec,
    grisms=['F115W-GR150R','F115W-GR150C','F150W-GR150R','F150W-GR150C','F200W-GR150R','F200W-GR150C'][0::2],
    size=24,
    grp=None,
    clean=False,
    get_cutout=1,
    cutout_filter=','.join(['F150W-CLEAR','F150WN-CLEAR','F200W-CLEAR','F200WN-CLEAR'][:]), # Direct image filters
    thumbnail_size=0.8 * 4,
    prefix=prefix,
    mb_kwargs={"min_sens": 0.0, "min_mask": 0.0},
    filter_kwargs=[{}, None][1],
    verbose=True,
    local=False,
    segmentation_image=segmentation_image,
    use_jwst_crds=True,
    **kwargs
)

https://grizli-cutout.herokuapp.com/exposures?polygon=rect(53.166632,-27.869037,0.25)&filters=F115W-GR150R,F150W-GR150R,F200W-GR150R&output=csv
extract_from_coords: dja-grism_033239.99-275208.53 27 exposures
./jw01283005001_04201_00001_nis_rate.fits exists
./jw01283005001_04201_00002_nis_rate.fits exists
./jw01283005001_04201_00003_nis_rate.fits exists
./jw01283005001_04201_00004_nis_rate.fits exists
./jw01283005001_04201_00005_nis_rate.fits exists
./jw01283005001_04201_00006_nis_rate.fits exists
./jw01283005001_04201_00007_nis_rate.fits exists
./jw01283005001_04201_00008_nis_rate.fits exists
./jw01283005001_04201_00009_nis_rate.fits exists
./jw01283005001_09201_00001_nis_rate.fits exists
./jw01283005001_09201_00002_nis_rate.fits exists
./jw01283005001_09201_00003_nis_rate.fits exists
./jw01283005001_09201_00004_nis_rate.fits exists
./jw01283005001_09201_00005_nis_rate.fits exists
./jw01283005001_09201_00006_nis_rate.fits exists
./jw01283005001_09201_00007_nis_rate.fits exists
./jw01283005001_09201_00008_nis_rate.fits exists
./jw01283005001_09201_00009_nis_rate.fits exists
./jw01283005001_14201_00001_nis_rate.fits exists
./jw01283005001_14201_00002_nis_rate.fits exists
./jw01283005001_14201_00003_nis_rate.fits exists
./jw01283005001_14201_00004_nis_rate.fits exists
./jw01283005001_14201_00005_nis_rate.fits exists
./jw01283005001_14201_00006_nis_rate.fits exists
./jw01283005001_14201_00007_nis_rate.fits exists
./jw01283005001_14201_00008_nis_rate.fits exists
./jw01283005001_14201_00009_nis_rate.fits exists
https://grizli-cutout.herokuapp.com/thumb?all_filters=False&filters=f150w-clear,f150wn-clear,f200w-clear,f200wn-clear&ra=53.1666323&dec=-27.8690371&size=3.2&output=fits_weight -> dja-grism_033239.99-275208.53_ir.fits
thumbnail: F150W-CLEAR
thumbnail: F150WN-CLEAR
thumbnail: F200W-CLEAR
thumbnail: F200WN-CLEAR
extract_from_coords: direct image = dja-grism_033239.99-275208.53_ir.fits
 1 / 27 GroupFLT jw01283005001_04201_00001_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
 2 / 27 GroupFLT jw01283005001_04201_00002_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
 3 / 27 GroupFLT jw01283005001_04201_00003_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
 4 / 27 GroupFLT jw01283005001_04201_00004_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
 5 / 27 GroupFLT jw01283005001_04201_00005_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
 6 / 27 GroupFLT jw01283005001_04201_00006_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
 7 / 27 GroupFLT jw01283005001_04201_00007_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
 8 / 27 GroupFLT jw01283005001_04201_00008_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
 9 / 27 GroupFLT jw01283005001_04201_00009_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0054.asdf GR150R F115W F115W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
10 / 27 GroupFLT jw01283005001_09201_00001_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
11 / 27 GroupFLT jw01283005001_09201_00002_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
12 / 27 GroupFLT jw01283005001_09201_00003_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
13 / 27 GroupFLT jw01283005001_09201_00004_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
14 / 27 GroupFLT jw01283005001_09201_00005_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
15 / 27 GroupFLT jw01283005001_09201_00006_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
16 / 27 GroupFLT jw01283005001_09201_00007_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
17 / 27 GroupFLT jw01283005001_09201_00008_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
18 / 27 GroupFLT jw01283005001_09201_00009_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0056.asdf GR150R F150W F150W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
19 / 27 GroupFLT jw01283005001_14201_00001_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
20 / 27 GroupFLT jw01283005001_14201_00002_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
21 / 27 GroupFLT jw01283005001_14201_00003_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
22 / 27 GroupFLT jw01283005001_14201_00004_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
23 / 27 GroupFLT jw01283005001_14201_00005_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
24 / 27 GroupFLT jw01283005001_14201_00006_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
25 / 27 GroupFLT jw01283005001_14201_00007_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
26 / 27 GroupFLT jw01283005001_14201_00008_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
27 / 27 GroupFLT jw01283005001_14201_00009_nis_rate.fits
get_conf: xxx /tmp/crds_cache/references/jwst/niriss/jwst_niriss_specwcs_0052.asdf GR150R F200W F200W None NIRISS
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
Using default C-based coordinate transformation...
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
extract_from_coords: dja-grism_033239.99-275208.53 27 beam cutouts

png

png

Fit redshift

This script tries to fit flexible functions to account for the contamination, and uses a simplified compact Gaussian profile to use for the optimal extraction. These are crude approximations, but actually tend to work acceptably for emission lines with reasonable S/N.

from importlib import reload
reload(sky_wfss)

dv = 800 # line width for fitting, km/s

b2d, zres = sky_wfss.combine_beams_2d(
    mb,
    step=0.5, pixfrac=0.75, # 2D "pseudodrizzle" parameters
    ymax=12.5,
    profile_sigma=1.5, profile_offset=-0.5, profile_type="gaussian",     # Gaussian cross-dispersion model
    # profile_type="grizli",                                             # Estimate a profile using the direct image
    bkg_percentile=None,
    # cont_spline=11, zfit_nspline=-1,  # Remove all contamination *before* fitting
    cont_spline=0, zfit_nspline=11,     # Fit with flexible splines to model contamination
    zfit_kwargs=dict(
        rest_wave=[6500, 4400], # Halpha - OIII
        velocity_sigma=dv,
        dz=dv/3.e5/2,
    ),
    ylim=(-3, 20), yticks=[0, 5, 10, 20],
    auto_niriss=False,
)


final spectrum file: dja-grism_033239.99-275208.53.f115w.spec.fits
final spectrum file: dja-grism_033239.99-275208.53.f150w.spec.fits
final spectrum file: dja-grism_033239.99-275208.53.f200w.spec.fits
redshift_fit_1d dja-grism_033239.99-275208.53 z=[0.492, 4.226] dz=0.0013333333333333333  nsteps=941
redshift_fit_1d dja-grism_033239.99-275208.53 best z=1.41168  dlnP=2526.9
redshift_fit_1d dja-grism_033239.99-275208.53 F115W dlnP=1375.5
redshift_fit_1d dja-grism_033239.99-275208.53 F150W dlnP=1151.6
redshift_fit_1d dja-grism_033239.99-275208.53 F200W dlnP=-0.1

png

png

png

png

Use original grizli tools to make line maps, measure line fluxes, etc.

Though it still doesn’t have a full contamination model….

templ = utils.load_templates(line_complexes=False, fwhm=1500)
splw = np.arange(9000, 3.e4, 5)
bspl = utils.bspline_templates(splw, df=31)
for t in templ:
    if t.startswith('line'):
        bspl[t] = templ[t]

tfit = mb.template_at_z(z=zres['z'], templates=bspl, fitter='lstsq')
_ = mb.drizzle_grisms_and_PAs(tfit=tfit, diff=True, kernel='point',)

if not os.path.exists('fit_args.npy'):
    from grizli.pipeline import auto_script
    fit_args = auto_script.generate_fit_params(include_photometry=False)


Saved arguments to fit_args.npy.

png

# Line map
# mb.drizzle_fit_lines?
# ! ls dja-grism_100014.21+021311.88*

dz = 0.01*(1+zres['z'])

from grizli import fitting, multifit
_ = fitting.run_all_parallel(
    0,
    file_pattern=mb.group_name,
    group_name=mb.group_name,
    get_output_data=True,
    fit_trace_shift=False,
    # zr=(zres['z'], ),                                 # Fix redshift
    zr=zres['z'] + np.array([-1,1])*0.01*(1+zres['z']), # Refit redshift in small range around best fit from above
    verbose=True,
    protect=False,
    # dscale=1./16/8, scale_linemap=4 / 4,
    min_sens=1.e-4, min_cont=1.e-4,
    t0=bspl,
    t1=bspl,
    pline={
        'kernel': 'square',
        'pixfrac': 0.5,
        'pixscale': 0.05,
        'size': 8,
        'wcs': None,
        # aligned with dispersion
        'theta': 270 - mb.beams[0].get_dispersion_PA(decimals=2),
    },
)

Run id=0 with fit_args.npy
load_master_fits: dja-grism_033239.99-275208.53.beams.fits
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
1 ./jw01283005001_04201_00001_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
2 ./jw01283005001_04201_00002_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
3 ./jw01283005001_04201_00003_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
4 ./jw01283005001_04201_00004_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
5 ./jw01283005001_04201_00005_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
6 ./jw01283005001_04201_00006_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
7 ./jw01283005001_04201_00007_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
8 ./jw01283005001_04201_00008_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
9 ./jw01283005001_04201_00009_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
10 ./jw01283005001_09201_00001_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
11 ./jw01283005001_09201_00002_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
12 ./jw01283005001_09201_00003_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
13 ./jw01283005001_09201_00004_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
14 ./jw01283005001_09201_00005_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
15 ./jw01283005001_09201_00006_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
16 ./jw01283005001_09201_00007_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
17 ./jw01283005001_09201_00008_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
18 ./jw01283005001_09201_00009_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
19 ./jw01283005001_14201_00001_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
20 ./jw01283005001_14201_00002_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
21 ./jw01283005001_14201_00003_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
22 ./jw01283005001_14201_00004_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
23 ./jw01283005001_14201_00005_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
24 ./jw01283005001_14201_00006_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
25 ./jw01283005001_14201_00007_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
26 ./jw01283005001_14201_00008_nis_rate.fits GR150R
load_new_sensitivity_curve: only defined for NIRCAM (NIRISS)
27 ./jw01283005001_14201_00009_nis_rate.fits GR150R
User templates! N=71 

Cache rest-frame template:  bspl 0 9000
Cache rest-frame template:  bspl 1 9340
Cache rest-frame template:  bspl 2 9830
Cache rest-frame template:  bspl 3 10500
Cache rest-frame template:  bspl 4 11250
Cache rest-frame template:  bspl 5 12000
Cache rest-frame template:  bspl 6 12750
Cache rest-frame template:  bspl 7 13500
Cache rest-frame template:  bspl 8 14250
Cache rest-frame template:  bspl 9 15000
Cache rest-frame template:  bspl 10 15750
Cache rest-frame template:  bspl 11 16500
Cache rest-frame template:  bspl 12 17250
Cache rest-frame template:  bspl 13 18000
Cache rest-frame template:  bspl 14 18750
Cache rest-frame template:  bspl 15 19495
Cache rest-frame template:  bspl 16 20245
Cache rest-frame template:  bspl 17 20995
Cache rest-frame template:  bspl 18 21745
Cache rest-frame template:  bspl 19 22495
Cache rest-frame template:  bspl 20 23245
Cache rest-frame template:  bspl 21 23995
Cache rest-frame template:  bspl 22 24745
Cache rest-frame template:  bspl 23 25495
Cache rest-frame template:  bspl 24 26245
Cache rest-frame template:  bspl 25 26995
Cache rest-frame template:  bspl 26 27745
Cache rest-frame template:  bspl 27 28495
Cache rest-frame template:  bspl 28 29165
Cache rest-frame template:  bspl 29 29655
Cache rest-frame template:  bspl 30 29995
  1.3876   40508.8 (1.3876) 1/6
  1.3971   38867.2 (1.3971) 2/6
  1.4067   34276.1 (1.4067) 3/6
  1.4164   30047.1 (1.4164) 4/6
  1.4261   35646.2 (1.4164) 5/6
  1.4358   39717.1 (1.4164) 6/6
First iteration: z_best=1.4164

zgrid_zoom: 1.4164 [1.4058, 1.4270] N=23
- 1.4058   34882.7 (1.4058) 1/23
- 1.4067   34287.2 (1.4067) 2/23
- 1.4077   33889.2 (1.4077) 3/23
- 1.4086   33474.8 (1.4086) 4/23
- 1.4096   33041.9 (1.4096) 5/23
- 1.4106   32585.3 (1.4106) 6/23
- 1.4115   31762.5 (1.4115) 7/23
- 1.4125   30673.6 (1.4125) 8/23
- 1.4135   30055.3 (1.4135) 9/23
- 1.4144   29995.7 (1.4144) 10/23
- 1.4154   30066.1 (1.4144) 11/23
- 1.4164   30048.4 (1.4144) 12/23
- 1.4173   30065.4 (1.4144) 13/23
- 1.4183   30625.2 (1.4144) 14/23
- 1.4193   31929.9 (1.4144) 15/23
- 1.4202   33209.7 (1.4144) 16/23
- 1.4212   33871.7 (1.4144) 17/23
- 1.4222   34117.9 (1.4144) 18/23
- 1.4231   34283.6 (1.4144) 19/23
- 1.4241   34515.7 (1.4144) 20/23
- 1.4251   34909.4 (1.4144) 21/23
- 1.4260   35624.6 (1.4144) 22/23
- 1.4270   36608.0 (1.4144) 23/23
Cache rest-frame template:  bspl 0 9000
Cache rest-frame template:  bspl 1 9340
Cache rest-frame template:  bspl 2 9830
Cache rest-frame template:  bspl 3 10500
Cache rest-frame template:  bspl 4 11250
Cache rest-frame template:  bspl 5 12000
Cache rest-frame template:  bspl 6 12750
Cache rest-frame template:  bspl 7 13500
Cache rest-frame template:  bspl 8 14250
Cache rest-frame template:  bspl 9 15000
Cache rest-frame template:  bspl 10 15750
Cache rest-frame template:  bspl 11 16500
Cache rest-frame template:  bspl 12 17250
Cache rest-frame template:  bspl 13 18000
Cache rest-frame template:  bspl 14 18750
Cache rest-frame template:  bspl 15 19495
Cache rest-frame template:  bspl 16 20245
Cache rest-frame template:  bspl 17 20995
Cache rest-frame template:  bspl 18 21745
Cache rest-frame template:  bspl 19 22495
Cache rest-frame template:  bspl 20 23245
Cache rest-frame template:  bspl 21 23995
Cache rest-frame template:  bspl 22 24745
Cache rest-frame template:  bspl 23 25495
Cache rest-frame template:  bspl 24 26245
Cache rest-frame template:  bspl 25 26995
Cache rest-frame template:  bspl 26 27745
Cache rest-frame template:  bspl 27 28495
Cache rest-frame template:  bspl 28 29165
Cache rest-frame template:  bspl 29 29655
Cache rest-frame template:  bspl 30 29995
Drizzle line -> SIII (1.54 0.35)
Drizzle line -> OII-7325 (0.90 0.35)
Drizzle line -> ArIII-7138 (8.61 12.55)
Drizzle line -> SII  (0.63 0.14)
Drizzle line -> Ha   (10.24 0.13)
Drizzle line -> OI-6302 (0.63 0.12)
Drizzle line -> HeI-5877 (0.57 0.10)
Drizzle line -> OIII (21.49 0.27)
Drizzle line -> Hb   (3.09 0.14)
Drizzle line -> OIII-4363 (1.29 0.63)
Drizzle line -> Hg   (0.09 0.76)
dja-grism_033239.99-275208.53_00000.full.fits

png

png

png

NIRCam

Example [OIII] / H$\alpha$ emitters with a PRISM spectrum.

ra, dec, prism_file = 150.1069005, 2.36004609, "cosmos-transients-v4_prism-clear_6585_61234.spec.fits"
ra, dec, prism_file = 150.09900755, 2.34362213, "glazebrook-cos-obs2-v4_prism-clear_2565_15420.spec.fits"
prism_file, ra, dec = 'gto-wide-cos02-v4_prism-clear_1214_727.spec.fits', 150.07850493, 2.35235787

# Show the prism spectrum
prism_mask = prism_file.split("_prism")[0]

Image(f"https://s3.amazonaws.com/msaexp-nirspec/extractions/{prism_mask}/{prism_file.replace('spec.fits', 'fnu.png')}")

png

mb = sky_wfss.extract_from_coords(
    ra=ra,
    dec=dec,
    grisms=['F356W-GRISMR','F430M-GRISMR','F410M-GRISMR','F444W-GRISMR','F444W-GRISMC','F480M-GRISMR','F480M-GRISMC'],
    size=48+32*1,
    grp=None,
    clean=False,
    get_cutout=1,
    cutout_filter=','.join(['F200W-CLEAR','F277W-CLEAR','F356W-CLEAR','F444W-CLEAR'][-3:]), #[::1][-1:]),
    thumbnail_size=0.8 * 2,
    prefix=prefix,
    mb_kwargs={},
    filter_kwargs=[{}, None][1],
    verbose=True,
    local=False,
    use_jwst_crds=False,
    **kwargs
)

https://grizli-cutout.herokuapp.com/exposures?polygon=rect(150.078505,2.352358,0.25)&filters=F356W-GRISMR,F430M-GRISMR,F410M-GRISMR,F444W-GRISMR,F444W-GRISMC,F480M-GRISMR,F480M-GRISMC&output=csv
extract_from_coords: dja-grism_100018.84+022108.49 10 exposures
./jw05893014003_02101_00001_nrcblong_rate.fits exists
./jw05893014004_02101_00003_nrcblong_rate.fits exists
./jw05893014004_02101_00004_nrcblong_rate.fits exists
./jw05893014004_02101_00005_nrcblong_rate.fits exists
./jw05893014004_02101_00006_nrcblong_rate.fits exists
./jw05893014003_02101_00002_nrcblong_rate.fits exists
./jw05893014003_02101_00003_nrcblong_rate.fits exists
./jw05893014003_02101_00004_nrcblong_rate.fits exists
./jw05893014003_02101_00005_nrcblong_rate.fits exists
./jw05893014003_02101_00006_nrcblong_rate.fits exists
https://grizli-cutout.herokuapp.com/thumb?all_filters=False&filters=f277w-clear,f356w-clear,f444w-clear&ra=150.07850493&dec=2.35235787&size=1.6&output=fits_weight -> dja-grism_100018.84+022108.49_ir.fits
thumbnail: F277W-CLEAR
thumbnail: F356W-CLEAR
thumbnail: F444W-CLEAR
extract_from_coords: direct image = dja-grism_100018.84+022108.49_ir.fits
 1 / 10 GroupFLT jw05893014003_02101_00001_nrcblong_rate.fits
Using default C-based coordinate transformation...
 2 / 10 GroupFLT jw05893014004_02101_00003_nrcblong_rate.fits
Using default C-based coordinate transformation...
 3 / 10 GroupFLT jw05893014004_02101_00004_nrcblong_rate.fits
Using default C-based coordinate transformation...
 4 / 10 GroupFLT jw05893014004_02101_00005_nrcblong_rate.fits
Using default C-based coordinate transformation...
 5 / 10 GroupFLT jw05893014004_02101_00006_nrcblong_rate.fits
Using default C-based coordinate transformation...
 6 / 10 GroupFLT jw05893014003_02101_00002_nrcblong_rate.fits
Using default C-based coordinate transformation...
 7 / 10 GroupFLT jw05893014003_02101_00003_nrcblong_rate.fits
Using default C-based coordinate transformation...
 8 / 10 GroupFLT jw05893014003_02101_00004_nrcblong_rate.fits
Using default C-based coordinate transformation...
 9 / 10 GroupFLT jw05893014003_02101_00005_nrcblong_rate.fits
Using default C-based coordinate transformation...
10 / 10 GroupFLT jw05893014003_02101_00006_nrcblong_rate.fits
Using default C-based coordinate transformation...
extract_from_coords: dja-grism_100018.84+022108.49 6 beam cutouts

png

png

Contamination or background over-subtraction can look troublesome, but can still find the line using the flexible contamination / continuum model

# Just show the simplified 1D extraction

from importlib import reload
reload(sky_wfss)

dv = 50 * 3

b2d, zres = sky_wfss.combine_beams_2d(
    mb,
    step=0.5, pixfrac=0.75,
    ymax=12.5,
    profile_sigma=1.5, profile_offset=-0.5, profile_type="gaussian",     # Gaussian cross-dispersion model
    # profile_type="grizli",                                   # Use the direct image thumbnail
    bkg_percentile=None,
    cont_spline=31*1, zfit_nspline=-1,
    zfit_kwargs=None,
)


final spectrum file: dja-grism_100018.84+022108.49.f444w.spec.fits

png

# Do the redshift fit

dv = 150 # km/s

b2d, zres = sky_wfss.combine_beams_2d(
    mb,
    step=0.5, pixfrac=0.75,
    ymax=12.5,
    profile_sigma=1.5, profile_offset=-0.5, profile_type="gaussian",     # Gaussian cross-dispersion model
    # profile_type="grizli",                                   # Use the direct image thumbnail
    bkg_percentile=None,
    cont_spline=31*1, zfit_nspline=-1,
    # cont_spline=0, zfit_nspline=31,
    zfit_kwargs=dict(
        rest_wave=[6800, 4800], # Halpha - OIII
        velocity_sigma=dv,
        dz=dv/3.e5/2,
    ),
)

final spectrum file: dja-grism_100018.84+022108.49.f444w.spec.fits
redshift_fit_1d dja-grism_100018.84+022108.49 z=[4.618, 9.582] dz=0.00025  nsteps=2534
redshift_fit_1d dja-grism_100018.84+022108.49 best z=5.25520  dlnP=113.8
redshift_fit_1d dja-grism_100018.84+022108.49 F444W dlnP=113.8

png

png

Compare to PRISM spectrum

# Prism spectrum
import msaexp.spectrum
prism = msaexp.spectrum.SpectrumSampler(
    f"https://s3.amazonaws.com/msaexp-nirspec/extractions/{prism_mask}/{prism_file}"
)


fig, axes = plt.subplots(1,2,figsize=(10, 4), sharey=True)
for ax in axes:
    ax.plot(prism['wave'], prism['flux'] * prism['to_flam'], label=prism_file, color='k', alpha=0.5)

    for gr in b2d:
        grism = b2d[gr]['spec']
        for PA in mb.PA[gr]:
            ix = mb.PA[gr][PA][0]
            break

        sens = np.interp(grism['wave']*1.e4, mb.beams[ix].beam.lam, mb.beams[ix].beam.sensitivity)
        trim = sens > 0.05*sens.max()
        ax.plot(grism['wave'][trim], (grism['flux'] / sens)[trim] * 1.e20, label=gr, alpha=0.5)

    ax.set_ylim(-5, 25)
    ax.grid()

for lrest in [5008, 6564.]:
    lobs = lrest * (1 + zres['z']) / 1.e4
    if lobs > grism['wave'].min():
        ax.set_xlim(lobs - 0.1, lobs + 0.1)
        break

axes[0].set_ylabel(r'$f_\lambda$ [$10^{-20}$ erg$~$/$~$s$~$/$~$cm$^2~$/$~\mathrm{\AA}$]')
# ax.set_xlim(3.8, 4.2)
# ax.set_xlim(4.5, 5.0)
fig.tight_layout(pad=1)

png

Line map

templ = utils.load_templates(line_complexes=False, fwhm=150)
splw = np.arange(9000, 3.e4, 5)
bspl = utils.bspline_templates(splw, df=31)
for t in templ:
    if t.startswith('line'):
        bspl[t] = templ[t]

tfit = mb.template_at_z(z=zres['z'], templates=bspl, fitter='lstsq')
_ = mb.drizzle_grisms_and_PAs(tfit=tfit, diff=True, kernel='point',)

if not os.path.exists('fit_args.npy'):
    from grizli.pipeline import auto_script
    fit_args = auto_script.generate_fit_params(include_photometry=False)

png

# Line map
# mb.drizzle_fit_lines?
# ! ls dja-grism_100014.21+021311.88*

fit_res = fitting.run_all_parallel(
    0,
    file_pattern=mb.group_name,
    group_name=mb.group_name,
    get_output_data=True,
    fit_trace_shift=False,
    zr=(zres['z'], ),                                 # Fix redshift
    # zr=zres['z'] + np.array([-1,1])*0.001*(1+zres['z']), # Refit redshift in small range around best fit from above
    verbose=True,
    protect=False,
    # dscale=1./16/8, scale_linemap=4 / 4,
    min_sens=1.e-4, min_cont=1.e-4,
    t0=bspl,
    t1=bspl,
    pline={
        'kernel': 'square',
        'pixfrac': 0.5,
        'pixscale': 0.05,
        'size': 8,
        'wcs': None,
        # aligned with dispersion
        'theta': 270 - mb.beams[0].get_dispersion_PA(decimals=2),
    },
)

Run id=0 with fit_args.npy
load_master_fits: dja-grism_100018.84+022108.49.beams.fits
1 ./jw05893014003_02101_00001_nrcblong_rate.fits F444W
2 ./jw05893014003_02101_00002_nrcblong_rate.fits F444W
3 ./jw05893014003_02101_00003_nrcblong_rate.fits F444W
4 ./jw05893014003_02101_00004_nrcblong_rate.fits F444W
5 ./jw05893014003_02101_00005_nrcblong_rate.fits F444W
6 ./jw05893014003_02101_00006_nrcblong_rate.fits F444W
User templates! N=71 

Cache rest-frame template:  bspl 0 9000
Cache rest-frame template:  bspl 1 9340
Cache rest-frame template:  bspl 2 9830
Cache rest-frame template:  bspl 3 10500
Cache rest-frame template:  bspl 4 11250
Cache rest-frame template:  bspl 5 12000
Cache rest-frame template:  bspl 6 12750
Cache rest-frame template:  bspl 7 13500
Cache rest-frame template:  bspl 8 14250
Cache rest-frame template:  bspl 9 15000
Cache rest-frame template:  bspl 10 15750
Cache rest-frame template:  bspl 11 16500
Cache rest-frame template:  bspl 12 17250
Cache rest-frame template:  bspl 13 18000
Cache rest-frame template:  bspl 14 18750
Cache rest-frame template:  bspl 15 19495
Cache rest-frame template:  bspl 16 20245
Cache rest-frame template:  bspl 17 20995
Cache rest-frame template:  bspl 18 21745
Cache rest-frame template:  bspl 19 22495
Cache rest-frame template:  bspl 20 23245
Cache rest-frame template:  bspl 21 23995
Cache rest-frame template:  bspl 22 24745
Cache rest-frame template:  bspl 23 25495
Cache rest-frame template:  bspl 24 26245
Cache rest-frame template:  bspl 25 26995
Cache rest-frame template:  bspl 26 27745
Cache rest-frame template:  bspl 27 28495
Cache rest-frame template:  bspl 28 29165
Cache rest-frame template:  bspl 29 29655
Cache rest-frame template:  bspl 30 29995
  5.2552  196564.9 (5.2552) 1/1
First iteration: z_best=5.2552

Cache rest-frame template:  bspl 0 9000
Cache rest-frame template:  bspl 1 9340
Cache rest-frame template:  bspl 2 9830
Cache rest-frame template:  bspl 3 10500
Cache rest-frame template:  bspl 4 11250
Cache rest-frame template:  bspl 5 12000
Cache rest-frame template:  bspl 6 12750
Cache rest-frame template:  bspl 7 13500
Cache rest-frame template:  bspl 8 14250
Cache rest-frame template:  bspl 9 15000
Cache rest-frame template:  bspl 10 15750
Cache rest-frame template:  bspl 11 16500
Cache rest-frame template:  bspl 12 17250
Cache rest-frame template:  bspl 13 18000
Cache rest-frame template:  bspl 14 18750
Cache rest-frame template:  bspl 15 19495
Cache rest-frame template:  bspl 16 20245
Cache rest-frame template:  bspl 17 20995
Cache rest-frame template:  bspl 18 21745
Cache rest-frame template:  bspl 19 22495
Cache rest-frame template:  bspl 20 23245
Cache rest-frame template:  bspl 21 23995
Cache rest-frame template:  bspl 22 24745
Cache rest-frame template:  bspl 23 25495
Cache rest-frame template:  bspl 24 26245
Cache rest-frame template:  bspl 25 26995
Cache rest-frame template:  bspl 26 27745
Cache rest-frame template:  bspl 27 28495
Cache rest-frame template:  bspl 28 29165
Cache rest-frame template:  bspl 29 29655
Cache rest-frame template:  bspl 30 29995


WARNING:py.warnings:/usr/local/lib/python3.11/dist-packages/grizli/fitting.py:4217: UserWarning: Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
  axz.set_xlim(zmi, zma)

2025-05-16 20:52:17,962 - stpipe - WARNING - /usr/local/lib/python3.11/dist-packages/grizli/fitting.py:4217: UserWarning: Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
  axz.set_xlim(zmi, zma)

WARNING:stpipe:/usr/local/lib/python3.11/dist-packages/grizli/fitting.py:4217: UserWarning: Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
  axz.set_xlim(zmi, zma)

WARNING:py.warnings:/usr/local/lib/python3.11/dist-packages/grizli/fitting.py:944: UserWarning: Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
  axz.set_xlim(zmi, zma)

2025-05-16 20:52:19,515 - stpipe - WARNING - /usr/local/lib/python3.11/dist-packages/grizli/fitting.py:944: UserWarning: Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
  axz.set_xlim(zmi, zma)

WARNING:stpipe:/usr/local/lib/python3.11/dist-packages/grizli/fitting.py:944: UserWarning: Attempting to set identical low and high xlims makes transformation singular; automatically expanding.
  axz.set_xlim(zmi, zma)



Drizzle line -> OII-7325 (-0.85 0.23)
Drizzle line -> ArIII-7138 (-0.41 0.18)
Drizzle line -> SII  (0.01 0.22)
Drizzle line -> Ha   (2.97 0.16)
Drizzle line -> OI-6302 (-1.05 0.19)
dja-grism_100018.84+022108.49_00000.full.fits

png

png

png

]]>Gabriel BrammerNIRSpec Merged Table2025-05-01T08:11:28+00:002025-05-01T08:11:28+00:00https://dawn-cph.github.io/dja/blog/2025/05/01/nirspec-merged-table-v4 spectroscopy nirspec release catalog

(This page is auto-generated from the Jupyter notebook nirspec-merged-table-v4.ipynb.)

Demo of full merged table of NIRSpec spectra reduced with msaexp. The merged columns are taken from the database tables

  • nirspec_extractions - Basic spectrum parameters (grating, mask, exposure time, etc.)
  • nirspec_redshifts - Redshift fit results, emission line fluxes
  • nirspec_redshifts_manual - Grades and comments from visual inspection
  • nirspec_integrated - Observed- and rest-frame filters integrated through the spectra at the derived redshift
  • grizli_photometry - Photometry and some eazy outputs of the nearest counterpart in the DJA/grizli photometric catalogs

The public spectra are shown in a large overview table at public_prelim_v4.2.html.

5 September 2025

The notebook has been updated to use a new version v4.4 of the merged table.

  • Includes PRISM spectra from the CANUCS GTO program, along with a number of additional public datasets
  • Merged files of the 1D spectra in all gratings are now available, with some grating examples added at the end of the demo.
  • The summary table and tables of compiled spectra now include all available spectra, whether or not they had been processed with the redshift fit algorithm.
  • Companion overview table at nirspec_public_v4.4.html. (Now has the same length as the summary table here.)
  • All files provided at the static DOI 10.5281/zenodo.1547235. See README.md at the zenodo page for more information on the release contents.

README

This repository is a snapshot of the public JWST NIRSpec spectra processed with the msaexp pipeline. Please refer to and cite de Graaff et al. (2024) and Heintz et al. (2025) for the main presentation of the msaexp pipeline.

This release corresponds to the v4 version of the spectral extractions, which significantly extends the wavelength range of the extracted spectra to regions that may suffer contamination of overlapping spectral orders. The sensitivity of the higher orders is strongly weighted toward the blue side of the spectrum for all gratings, so, in practice, relatively red galaxies often suffer relatively minor order contamination.

Please refer to and cite this DOI 10.5281/zenodo.1547235 and Valentino et al. (2025) when using this specific data release and the v4 spectra for a presentation of the extended extractions. For more information and updates, please refer to the DJA Blog Post: https://dawn-cph.github.io/dja/blog/2025/05/01/nirspec-merged-table-v4/

Data Content

This release provides version 4 of the NIRSpec Merged Table, a comprehensive catalog of uniformly reduced and analyzed JWST/NIRSpec spectra. The data have been processed using the msaexp and grizli pipelines and are publicly available through the DAWN JWST Archive (DJA). The catalog integrates multiple data products, including spectral extractions, redshift measurements, emission line fluxes, and photometric associations.

The merged table consolidates information from several database tables:

  • nirspec_extractions: Basic spectrum parameters (e.g., grating, mask, exposure time).
  • nirspec_redshifts: Redshift fit results and emission line fluxes.
  • nirspec_redshifts_manual: Grades and comments from visual inspection.
  • nirspec_integrated: Observed- and rest-frame filters integrated through the spectra at the derived redshift.
  • grizli_photometry: Photometry and some EAZY outputs of the nearest counterpart in the DJA/grizli photometric catalogs.

The catalog includes 80,367 entries, each corresponding to a unique NIRSpec spectrum. Each entry contains metadata such as source ID, coordinates, grating/filter configuration, exposure time, redshift estimates, emission line measurements, and photometric associations.

N Grating-Filter Concatenated 1D spectrum file
113 G140H-F070LP dja_msaexp_emission_lines_v4.4.g140h-f070lp_spectra.fits
684 G140H-F100LP dja_msaexp_emission_lines_v4.4.g140h-f100lp_spectra.fits
5851 G140M-F070LP dja_msaexp_emission_lines_v4.4.g140m-f070lp_spectra.fits
2165 G140M-F100LP dja_msaexp_emission_lines_v4.4.g140m-f100lp_spectra.fits
6179 G235H-F170LP dja_msaexp_emission_lines_v4.4.g235h-f170lp_spectra.fits
8000 G235M-F170LP dja_msaexp_emission_lines_v4.4.g235m-f170lp_spectra.fits
8820 G395H-F290LP dja_msaexp_emission_lines_v4.4.g395h-f290lp_spectra.fits
13606 G395M-F290LP dja_msaexp_emission_lines_v4.4.g395m-f290lp_spectra.fits
34949 PRISM-CLEAR dja_msaexp_emission_lines_v4.4.prism_spectra.fits

Usage Notes

  • Data Format: The main catalog is provided in compressed CSV format. Column descriptions, including units and formats, are detailed in the accompanying columns CSV file.

The redshift and line fluxes are derived with the functions msaexp.fit_redshift and msaexp.plot_spectrum. The continuum is modelled as a combination of splines, whose coefficient are listed in the table. Emission lines are superimposed as Gaussian profiles, smoothed by the instrumental resolution (increased by a factor of 1.3x compared with the nominal line spread functions in the JWST User Documentation, JDox, see de Graaff+2025) and a fixed line velocity width of 100 km/s. Equivalent widths in angstrom in the observed frame are also reported. A dictionary with the available lines and their rest-frame wavelengths in vacuum can be generated with grizli.utils.get_line_wavelengths(). The fit is performed with a least square template method. The uncertainties are rescaled with a polynomial curve prior to fitting such that (flux - model) / (err * scl) residuals are N(0,1) using msaexp.spectrum.calc_uncertainty_scale. A systematic uncertainty floor of 2% is introduced. The robustness of the redshift solution is flagged with visual inspection according to the following scheme:

  • Grade 3: Robust redshift from one or more emission absorption features
  • Grade 2: Ambiguous continuum features, perhaps only one line or low confidence lines
  • Grade 1: No clear features in the spectrum to constrain the redshift
  • Grade 0: Spectrum suffers some data quality issue and should
  • Grade -1: Fit not performed or graded

If multiple spectra of the same sources are available, a common best redshift solution is stored in the z_best column.

  • Spectral Data: Each entry includes links to the corresponding 2D spectra in fits and png format, which can be accessed through the DJA interface or the public spectra overview page at: nirspec_public_v4.4.html The 1D spectra in the fits tables in this release are in the format generated by msaexp. The columns are described in dja_msaexp_emission_lines_v4.4.columns.csv.

  • Some masks that provide spectra of predominantly local sources in the MW or in nearby galaxies were not processed through the redshift fitting procedure. The general spectrum metadata is included in the summary table and the spectra are include in the compiled tables, but the columns of quantities derived from the redshift fit (e.g., emission line fluxes) are not populated.

  • If existing, a comparison with the previous v3 version the spectra generated with msaexp is available at the same link on the DJA interface.

  • Photometric Associations: Photometric data are matched to the nearest counterparts in the DJA/grizli catalogs, providing additional context for each spectroscopic observation. The latest public versions of the DJA/grizli catalogs can are described here: https://dawn-cph.github.io/dja/imaging/v7/ The latest v7* version of the imaging mosaics and catalogs can be retrieved here: https://s3.amazonaws.com/grizli-v2/JwstMosaics/v7/index.html A description of the data reduction and catalog creation is available on the DJA interface and in Valentino+2023.

Caveats

  • An effective extended-source path-loss correction for light outside of the slitlet for each source using the a priori position within the shutter and assuming an azimuthally symmetric Gaussian profile is applied to each spectrum in this release (de Graaff et al. 2025, Section 3). However, the spectra are not rescaled to match the observed photometry in the “phot_” columns.
  • The flux calibration is derived from calibration, monitoring, and scientific programs (Valentino et al. 2025). However, residual features in spectra due to an imperfect cross-calibration of different overlapping orders are still present in the released spectra.
  • Examples of second order corrections are presented in Ito et al. (2025), where medium-resolution grating spectra beyond their nominal coverage deviate from the prism counterpart (Figure C.1 in appendix in Ito et al. 2025). Moreover, a significant downturn is present in prism spectra at wavelengths longer than 5.2 µm. These second order corrections are largely mitigated cross-calibrating all the available spectra and photometry by means of simple low-order polynomial corrections, for example available in spectrophotometric modeling codes. Future calibration programs dedicated to reconstruction of the sensitivity curves in the extended spectra at different location of the MSA will allow for refined absolute calibrations.

Software

The data processing and analysis utilized the following software packages:

  • msaexp: v0.9.8.dev3+ge0e3f39.d20250429
  • grizli: v1.12.12.dev5+g5896d62.d20250426
  • eazy-py: v0.8.5

Citations

We encourage users to refer to the original works describing the datasets collected in this release. Relevant references, compiled to the best of our knowledge, are available here: https://dawn-cph.github.io/dja/spectroscopy/nirspec/. Please file an Issue on the DJA website repository if you would prefer a different citation for your own dataset.

Works that make use of the products of the DJA should cite this DOI and relevant articles describing them:

1) Msaexp pipeline and methods, v3 and previous spectroscopic compilation releases:

  • de Graaff, A., Brammer, G., Weibel, A., et al., “RUBIES: a complete census of the bright and red distant Universe with JWST/NIRSpec”, A&A, 697, 189 (2025)
  • Heintz, K. E., Brammer, G., Watson, D., et al., “The JWST-PRIMAL archival survey: A JWST/NIRSpec reference sample for the physical properties and Lyman-α absorption and emission of ∼600 galaxies at z = 5.0−13.4”, A&A, 693, 60 (2025)
  • Brammer G., “msaexp: NIRSpec analyis tools”, 10.5281/zenodo.7299500 (2022)

2) Spectroscopic release v4 and extended spectra:

  • Valentino, F., Heintz, K. E., Brammer, G. et al., “Gas outflows in two recently quenched galaxies at z = 4 and 7”, A&A, 699, 358 (2025)
  • Pollock, C., Gottumukkala, R., Heintz, K. E. et al., “Novel z~10 auroral line measurements extend the gradual offset of the FMR deep into the first Gyr of cosmic time “, arXiv:2506.15779 (2025)

3) Grizli pipeline:

  • Brammer G., “grizli”, https://zenodo.org/records/8370018 (2023)

4) Imaging release:

  • Valentino, F., Brammer, G., Gould, K. M. L. et al., “An Atlas of Color-selected Quiescent Galaxies at z > 3 in Public JWST Fields”, ApJ, 947, 20 (2023)

Demo

# Install dependencies, e.g., on Google Colab
try:
    import msaexp

except ImportError:

    ! pip install msaexp
    ! pip install git+https://github.com/karllark/dust_attenuation.git
    
    import eazy
    eazy.fetch_eazy_photoz()

%matplotlib inline

import os
import yaml

import numpy as np
import matplotlib.pyplot as plt
from tqdm import tqdm

import warnings
warnings.filterwarnings('ignore')

from scipy.spatial import cKDTree

import astropy.io.fits as pyfits
from astropy.utils.data import download_file
from astropy.cosmology import WMAP9
import astropy.units as u

import grizli
import grizli.catalog
from grizli import utils

import eazy
import msaexp

CACHE_DOWNLOADS = True

print(f'grizli version: {grizli.__version__}')
print(f'eazy-py version: {eazy.__version__}')
print(f'msaexp version: {msaexp.__version__}')

grizli version: 1.13.2.dev1+g1439c13c5.d20250908
eazy-py version: 0.8.5
msaexp version: 0.9.12.dev20+g2d24ffa54.d20250903

Read the table

# Full table
version = "v4.0" # Original notebook release
version = "v4.3" # Updated August 11, 2025.  Includes CANUCS and some other additional masks.
version = "v4.4" # Updated September 5, 2025.  Include all public spectra even without redshift / line fits

URL_PREFIX = "https://s3.amazonaws.com/msaexp-nirspec/extractions"

# Use the Zenodo release
if version == "v4.4":
    URL_PREFIX = "https://zenodo.org/records/15472354/files/"

table_url = f"{URL_PREFIX}/dja_msaexp_emission_lines_{version}.csv.gz"

tab = utils.read_catalog(download_file(table_url, cache=CACHE_DOWNLOADS), format='csv')

Column descriptions

columns_url = f"{URL_PREFIX}/dja_msaexp_emission_lines_{version}.columns.csv"
tab_columns = utils.read_catalog(download_file(columns_url, cache=CACHE_DOWNLOADS), format='csv')

# Set column metadata
for row in tab_columns:
    c = row['column']
    if row['unit'] != '--':
        tab[c].unit = row['unit']
    if row['format'] != '--':
        tab[c].format = row['format']
    if row['description'] != '--':
        tab[c].description = row['description']

tab.info()

<GTable length=80367>
        name         dtype   unit  format                                       description                                           class     n_bad
------------------- ------- ------ ------ ---------------------------------------------------------------------------------------- ------------ -----
               file   str57                                                                                           DJA filename       Column     0
              srcid   int64                                                                                Source ID from APT plan       Column     0
                 ra float64    deg    .8f                                                                         RA from APT plan       Column     0
                dec float64    deg    .8f                                                                        Dec from APT plan       Column     0
            grating    str5                                                                                        NIRSpec grating       Column     0
             filter    str6                                                                                        Blocking filter       Column     0
           effexptm float64                                                               Effective exposure time of each exposure       Column     0
             nfiles   int64                                                             Number of files combined in final spectrum       Column     0
            dataset   str72                                                                             Filename of first exposure       Column     0
             msamet   str25                                                                                      MSA metadata file       Column     0
              msaid   int64                                                                                        MSA metadata ID       Column     0
             msacnf   int64                                                                                    MSA metadata config       Column     0
              dithn   int64                                                                                          Dither number       Column     0
             slitid   int64                                                                                       MSA plan slit ID       Column     0
               root   str24                                                                            DJA program + mask rootname       Column     0
               npix   int64                                                                    Number of pixels in the 2D spectrum       Column     0
               ndet   int64                                                             Number of detectors contributing to output       Column     0
               wmin float64 micron                                                     Minimum wavelength of the combined spectrum       Column     0
               wmax float64 micron                                                     Maximum wavelength of the combined spectrum       Column     0
             wmaxsn float64 micron                                                           Wavelength of maximum signal to noise       Column     0
               sn10 float64                                                                                     10th percentile SN       Column     0
             flux10 float64                                                                                           Flux at sn10       Column     0
              err10 float64                                                                                    Uncertainty at sn50       Column     0
               sn50 float64                                                                                     50th percentile SN       Column     0
             flux50 float64                                                                                           Flux at sn50       Column     0
              err50 float64                                                                                    Uncertainty at sn90       Column     0
               sn90 float64                                                                                     90th percentile SN       Column     0
             flux90 float64                                                                                           Flux at sn90       Column     0
              err90 float64                                                                                    Uncertainty at sn90       Column     0
             xstart   int64                                                            Starting detector x coordinate of 2D cutout       Column     0
             ystart   int64                                                            Starting detector y coordinate of 2D cutout       Column     0
              xsize   int64                                                                                    x size of 2D cutout       Column     0
              ysize   int64                                                                                    y size of 2D cutout       Column     0
            slit_pa float64                                                                            Estimated PA of the slitlet       Column     0
              pa_v3 float64                                                                 Estimated PA of the spacecraft V3 axis       Column     0
            srcypix float64                                                              Location of the source in the 2D spectrum       Column     0
            profcen float64                                                         Profile offset relative to the expected center       Column     0
            profsig float64                                                           Derived profile width in pixels added to PSF       Column     0
              ctime float64                                                                      UNIX time when file was generated       Column     0
            version   str30                                                                                    MSAEXP code version       Column     0
            exptime float64                                                                          Estimated total exposure time       Column     0
           contchi2 float64                                                                       Chi2 of the spline continuum fit MaskedColumn 18242
                dof   int64                                                      Total number of pixels in the redshift + line fit MaskedColumn 18242
           fullchi2 float64                                                                  Chi2 of the full continuum + line fit MaskedColumn 18242
    line_ariii_7138 float64                                                                Line flux of ariii_7138 1e-20 erg/s/cm2 MaskedColumn 35864
line_ariii_7138_err float64                                                                                                        MaskedColumn 35868
    line_ariii_7753 float64                                                                Line flux of ariii_7753 1e-20 erg/s/cm2 MaskedColumn 37381
line_ariii_7753_err float64                                                                                                        MaskedColumn 37382
           line_bra float64                                                                       Line flux of bra 1e-20 erg/s/cm2 MaskedColumn 77651
       line_bra_err float64                                                                                                        MaskedColumn 77651
           line_brb float64                                                                       Line flux of brb 1e-20 erg/s/cm2 MaskedColumn 71170
       line_brb_err float64                                                                                                        MaskedColumn 71170
           line_brd float64                                                                       Line flux of brd 1e-20 erg/s/cm2 MaskedColumn 64722
       line_brd_err float64                                                                                                        MaskedColumn 64722
           line_brg float64                                                                       Line flux of brg 1e-20 erg/s/cm2 MaskedColumn 67432
       line_brg_err float64                                                                                                        MaskedColumn 67432
            line_hb float64                                                                        Line flux of hb 1e-20 erg/s/cm2 MaskedColumn 36409
        line_hb_err float64                                                                                                        MaskedColumn 36409
            line_hd float64                                                                        Line flux of hd 1e-20 erg/s/cm2 MaskedColumn 40270
        line_hd_err float64                                                                                                        MaskedColumn 40270
      line_hei_1083 float64                                                                  Line flux of hei_1083 1e-20 erg/s/cm2 MaskedColumn 46220
  line_hei_1083_err float64                                                                                                        MaskedColumn 46223
      line_hei_3889 float64                                                                  Line flux of hei_3889 1e-20 erg/s/cm2 MaskedColumn 51316
  line_hei_3889_err float64                                                                                                        MaskedColumn 51319
      line_hei_5877 float64                                                                  Line flux of hei_5877 1e-20 erg/s/cm2 MaskedColumn 34474
  line_hei_5877_err float64                                                                                                        MaskedColumn 34474
      line_hei_7065 float64                                                                  Line flux of hei_7065 1e-20 erg/s/cm2 MaskedColumn 35664
  line_hei_7065_err float64                                                                                                        MaskedColumn 35669
      line_hei_8446 float64                                                                  Line flux of hei_8446 1e-20 erg/s/cm2 MaskedColumn 39420
  line_hei_8446_err float64                                                                                                        MaskedColumn 39420
     line_heii_4687 float64                                                                 Line flux of heii_4687 1e-20 erg/s/cm2 MaskedColumn 59306
 line_heii_4687_err float64                                                                                                        MaskedColumn 59306
            line_hg float64                                                                        Line flux of hg 1e-20 erg/s/cm2 MaskedColumn 38897
        line_hg_err float64                                                                                                        MaskedColumn 38897
           line_lya float64                                                                       Line flux of lya 1e-20 erg/s/cm2 MaskedColumn 68665
       line_lya_err float64                                                                                                        MaskedColumn 68665
          line_mgii float64                                                                      Line flux of mgii 1e-20 erg/s/cm2 MaskedColumn 49321
      line_mgii_err float64                                                                                                        MaskedColumn 49321
    line_neiii_3867 float64                                                                Line flux of neiii_3867 1e-20 erg/s/cm2 MaskedColumn 61857
line_neiii_3867_err float64                                                                                                        MaskedColumn 61858
    line_neiii_3968 float64                                                                Line flux of neiii_3968 1e-20 erg/s/cm2 MaskedColumn 41226
line_neiii_3968_err float64                                                                                                        MaskedColumn 41228
      line_nev_3346 float64                                                                  Line flux of nev_3346 1e-20 erg/s/cm2 MaskedColumn 45488
  line_nev_3346_err float64                                                                                                        MaskedColumn 45488
     line_nevi_3426 float64                                                                 Line flux of nevi_3426 1e-20 erg/s/cm2 MaskedColumn 44940
 line_nevi_3426_err float64                                                                                                        MaskedColumn 44941
     line_niii_1750 float64                                                                 Line flux of niii_1750 1e-20 erg/s/cm2 MaskedColumn 59941
 line_niii_1750_err float64                                                                                                        MaskedColumn 59941
       line_oi_6302 float64                                                                   Line flux of oi_6302 1e-20 erg/s/cm2 MaskedColumn 34327
   line_oi_6302_err float64                                                                                                        MaskedColumn 34327
           line_oii float64                                                                       Line flux of oii 1e-20 erg/s/cm2 MaskedColumn 42821
      line_oii_7325 float64                                                                  Line flux of oii_7325 1e-20 erg/s/cm2 MaskedColumn 50553
  line_oii_7325_err float64                                                                                                        MaskedColumn 50554
       line_oii_err float64                                                                                                        MaskedColumn 42822
          line_oiii float64                                                 Line flux of combined OIII 4959+5007 1e-20 erg/s/cm2/A MaskedColumn 57718
     line_oiii_1663 float64                                                                 Line flux of oiii_1663 1e-20 erg/s/cm2 MaskedColumn 61062
 line_oiii_1663_err float64                                                                                                        MaskedColumn 61062
     line_oiii_4363 float64                                                                 Line flux of oiii_4363 1e-20 erg/s/cm2 MaskedColumn 60229
 line_oiii_4363_err float64                                                                                                        MaskedColumn 60229
     line_oiii_4959 float64                                                                 Line flux of oiii_4959 1e-20 erg/s/cm2 MaskedColumn 58726
 line_oiii_4959_err float64                                                                                                        MaskedColumn 58726
     line_oiii_5007 float64                                                                 Line flux of oiii_5007 1e-20 erg/s/cm2 MaskedColumn 58598
 line_oiii_5007_err float64                                                                                                        MaskedColumn 58598
      line_oiii_err float64                                                                                                        MaskedColumn 57718
          line_pa10 float64                                                                      Line flux of pa10 1e-20 erg/s/cm2 MaskedColumn 41147
      line_pa10_err float64                                                                                                        MaskedColumn 41152
           line_pa8 float64                                                                       Line flux of pa8 1e-20 erg/s/cm2 MaskedColumn 42686
       line_pa8_err float64                                                                                                        MaskedColumn 42691
           line_pa9 float64                                                                       Line flux of pa9 1e-20 erg/s/cm2 MaskedColumn 41764
       line_pa9_err float64                                                                                                        MaskedColumn 41767
           line_paa float64                                                                       Line flux of paa 1e-20 erg/s/cm2 MaskedColumn 63659
       line_paa_err float64                                                                                                        MaskedColumn 63659
           line_pab float64                                                                       Line flux of pab 1e-20 erg/s/cm2 MaskedColumn 52440
       line_pab_err float64                                                                                                        MaskedColumn 52441
           line_pad float64                                                                       Line flux of pad 1e-20 erg/s/cm2 MaskedColumn 44197
       line_pad_err float64                                                                                                        MaskedColumn 44201
           line_pag float64                                                                       Line flux of pag 1e-20 erg/s/cm2 MaskedColumn 46554
       line_pag_err float64                                                                                                        MaskedColumn 46557
           line_pfb float64                                                                       Line flux of pfb 1e-20 erg/s/cm2 MaskedColumn 79212
       line_pfb_err float64                                                                                                        MaskedColumn 79212
           line_pfd float64                                                                       Line flux of pfd 1e-20 erg/s/cm2 MaskedColumn 74917
       line_pfd_err float64                                                                                                        MaskedColumn 74918
           line_pfe float64                                                                       Line flux of pfe 1e-20 erg/s/cm2 MaskedColumn 73676
       line_pfe_err float64                                                                                                        MaskedColumn 73676
           line_pfg float64                                                                       Line flux of pfg 1e-20 erg/s/cm2 MaskedColumn 76553
       line_pfg_err float64                                                                                                        MaskedColumn 76553
           line_sii float64                                                                       Line flux of sii 1e-20 erg/s/cm2 MaskedColumn 49475
       line_sii_err float64                                                                                                        MaskedColumn 49480
     line_siii_9068 float64                                                                 Line flux of siii_9068 1e-20 erg/s/cm2 MaskedColumn 41324
 line_siii_9068_err float64                                                                                                        MaskedColumn 41329
     line_siii_9531 float64                                                                 Line flux of siii_9531 1e-20 erg/s/cm2 MaskedColumn 42649
 line_siii_9531_err float64                                                                                                        MaskedColumn 42654
              spl_0 float64                                                                         Spline continuum coefficient 0 MaskedColumn 18242
          spl_0_err float64                                                                                                        MaskedColumn 18242
              spl_1 float64                                                                         Spline continuum coefficient 1 MaskedColumn 18242
             spl_10 float64                                                                        Spline continuum coefficient 10 MaskedColumn 18242
         spl_10_err float64                                                                                                        MaskedColumn 18242
             spl_11 float64                                                                        Spline continuum coefficient 11 MaskedColumn 18242
         spl_11_err float64                                                                                                        MaskedColumn 18243
             spl_12 float64                                                                        Spline continuum coefficient 12 MaskedColumn 18242
         spl_12_err float64                                                                                                        MaskedColumn 18245
             spl_13 float64                                                                        Spline continuum coefficient 13 MaskedColumn 18242
         spl_13_err float64                                                                                                        MaskedColumn 18244
             spl_14 float64                                                                        Spline continuum coefficient 14 MaskedColumn 18242
         spl_14_err float64                                                                                                        MaskedColumn 18244
             spl_15 float64                                                                        Spline continuum coefficient 15 MaskedColumn 18242
         spl_15_err float64                                                                                                        MaskedColumn 18244
             spl_16 float64                                                                        Spline continuum coefficient 16 MaskedColumn 18242
         spl_16_err float64                                                                                                        MaskedColumn 18243
             spl_17 float64                                                                        Spline continuum coefficient 17 MaskedColumn 18242
         spl_17_err float64                                                                                                        MaskedColumn 18243
             spl_18 float64                                                                        Spline continuum coefficient 18 MaskedColumn 18242
         spl_18_err float64                                                                                                        MaskedColumn 18242
             spl_19 float64                                                                        Spline continuum coefficient 19 MaskedColumn 18242
         spl_19_err float64                                                                                                        MaskedColumn 18242
          spl_1_err float64                                                                                                        MaskedColumn 18242
              spl_2 float64                                                                         Spline continuum coefficient 2 MaskedColumn 18242
             spl_20 float64                                                                        Spline continuum coefficient 20 MaskedColumn 18242
         spl_20_err float64                                                                                                        MaskedColumn 18242
             spl_21 float64                                                                        Spline continuum coefficient 21 MaskedColumn 18242
         spl_21_err float64                                                                                                        MaskedColumn 18242
             spl_22 float64                                                                        Spline continuum coefficient 22 MaskedColumn 18242
         spl_22_err float64                                                                                                        MaskedColumn 18242
          spl_2_err float64                                                                                                        MaskedColumn 18242
              spl_3 float64                                                                         Spline continuum coefficient 3 MaskedColumn 18242
          spl_3_err float64                                                                                                        MaskedColumn 18242
              spl_4 float64                                                                         Spline continuum coefficient 4 MaskedColumn 18242
          spl_4_err float64                                                                                                        MaskedColumn 18242
              spl_5 float64                                                                         Spline continuum coefficient 5 MaskedColumn 18242
          spl_5_err float64                                                                                                        MaskedColumn 18243
              spl_6 float64                                                                         Spline continuum coefficient 6 MaskedColumn 18242
          spl_6_err float64                                                                                                        MaskedColumn 18243
              spl_7 float64                                                                         Spline continuum coefficient 7 MaskedColumn 18242
          spl_7_err float64                                                                                                        MaskedColumn 18247
              spl_8 float64                                                                         Spline continuum coefficient 8 MaskedColumn 18242
          spl_8_err float64                                                                                                        MaskedColumn 18243
              spl_9 float64                                                                         Spline continuum coefficient 9 MaskedColumn 18242
          spl_9_err float64                                                                                                        MaskedColumn 18245
              zline float64                                                                     Redshift where the lines where fit MaskedColumn 18242
      line_civ_1549 float64                                                                  Line flux of civ_1549 1e-20 erg/s/cm2 MaskedColumn 62960
  line_civ_1549_err float64                                                                                                        MaskedColumn 62960
           line_h10 float64                                                                       Line flux of h10 1e-20 erg/s/cm2 MaskedColumn 71081
       line_h10_err float64                                                                                                        MaskedColumn 71081
           line_h11 float64                                                                       Line flux of h11 1e-20 erg/s/cm2 MaskedColumn 71209
       line_h11_err float64                                                                                                        MaskedColumn 71209
           line_h12 float64                                                                       Line flux of h12 1e-20 erg/s/cm2 MaskedColumn 71254
       line_h12_err float64                                                                                                        MaskedColumn 71254
            line_h7 float64                                                                        Line flux of h7 1e-20 erg/s/cm2 MaskedColumn 70536
        line_h7_err float64                                                                                                        MaskedColumn 70536
            line_h8 float64                                                                        Line flux of h8 1e-20 erg/s/cm2 MaskedColumn 70806
        line_h8_err float64                                                                                                        MaskedColumn 70806
            line_h9 float64                                                                        Line flux of h9 1e-20 erg/s/cm2 MaskedColumn 70958
        line_h9_err float64                                                                                                        MaskedColumn 70958
            line_ha float64                                                                        Line flux of ha 1e-20 erg/s/cm2 MaskedColumn 65641
        line_ha_err float64                                                                                                        MaskedColumn 65641
      line_hei_6680 float64                                                                  Line flux of hei_6680 1e-20 erg/s/cm2 MaskedColumn 65749
  line_hei_6680_err float64                                                                                                        MaskedColumn 65749
     line_heii_1640 float64                                                                 Line flux of heii_1640 1e-20 erg/s/cm2 MaskedColumn 61662
 line_heii_1640_err float64                                                                                                        MaskedColumn 61662
      line_nii_6549 float64                                                                  Line flux of nii_6549 1e-20 erg/s/cm2 MaskedColumn 65634
  line_nii_6549_err float64                                                                                                        MaskedColumn 65634
      line_nii_6584 float64                                                                  Line flux of nii_6584 1e-20 erg/s/cm2 MaskedColumn 65646
  line_nii_6584_err float64                                                                                                        MaskedColumn 65646
      line_oii_7323 float64                                                                  Line flux of oii_7323 1e-20 erg/s/cm2 MaskedColumn 66230
  line_oii_7323_err float64                                                                                                        MaskedColumn 66230
      line_oii_7332 float64                                                                  Line flux of oii_7332 1e-20 erg/s/cm2 MaskedColumn 66244
  line_oii_7332_err float64                                                                                                        MaskedColumn 66244
      line_sii_6717 float64                                                                  Line flux of sii_6717 1e-20 erg/s/cm2 MaskedColumn 65776
  line_sii_6717_err float64                                                                                                        MaskedColumn 65776
      line_sii_6731 float64                                                                  Line flux of sii_6731 1e-20 erg/s/cm2 MaskedColumn 65760
  line_sii_6731_err float64                                                                                                        MaskedColumn 65760
     line_siii_6314 float64                                                                 Line flux of siii_6314 1e-20 erg/s/cm2 MaskedColumn 65692
 line_siii_6314_err float64                                                                                                        MaskedColumn 65692
            escale0 float64                                                             0th coefficient of the uncertainty scaling MaskedColumn 18242
            escale1 float64                                                             1st coefficient of the uncertainty scaling MaskedColumn 18242
     line_ciii_1906 float64                                                                 Line flux of ciii_1906 1e-20 erg/s/cm2 MaskedColumn 57937
 line_ciii_1906_err float64                                                                                                        MaskedColumn 57937
      line_niv_1487 float64                                                                  Line flux of niv_1487 1e-20 erg/s/cm2 MaskedColumn 65762
  line_niv_1487_err float64                                                                                                        MaskedColumn 65762
      line_pah_3p29 float64                                                                  Line flux of pah_3p29 1e-20 erg/s/cm2 MaskedColumn 74940
  line_pah_3p29_err float64                                                                                                        MaskedColumn 74940
      line_pah_3p40 float64                                                                  Line flux of pah_3p40 1e-20 erg/s/cm2 MaskedColumn 74940
  line_pah_3p40_err float64                                                                                                        MaskedColumn 74940
     eqw_ariii_7138 float64                                                          Observed-frame equivalent width in ariii_7138 MaskedColumn 35897
     eqw_ariii_7753 float64                                                          Observed-frame equivalent width in ariii_7753 MaskedColumn 37422
            eqw_bra float64                                                                 Observed-frame equivalent width in bra MaskedColumn 77651
            eqw_brb float64                                                                 Observed-frame equivalent width in brb MaskedColumn 71180
            eqw_brd float64                                                                 Observed-frame equivalent width in brd MaskedColumn 64731
            eqw_brg float64                                                                 Observed-frame equivalent width in brg MaskedColumn 67448
      eqw_ciii_1906 float64                                                           Observed-frame equivalent width in ciii_1906 MaskedColumn 57948
       eqw_civ_1549 float64                                                            Observed-frame equivalent width in civ_1549 MaskedColumn 62963
         eqw_ha_nii float64                                                              Observed-frame equivalent width in ha_nii MaskedColumn 49293
             eqw_hb float64                                                                  Observed-frame equivalent width in hb MaskedColumn 36439
             eqw_hd float64                                                                  Observed-frame equivalent width in hd MaskedColumn 40299
       eqw_hei_1083 float64                                                            Observed-frame equivalent width in hei_1083 MaskedColumn 46257
       eqw_hei_3889 float64                                                            Observed-frame equivalent width in hei_3889 MaskedColumn 51346
       eqw_hei_5877 float64                                                            Observed-frame equivalent width in hei_5877 MaskedColumn 34498
       eqw_hei_7065 float64                                                            Observed-frame equivalent width in hei_7065 MaskedColumn 35702
       eqw_hei_8446 float64                                                            Observed-frame equivalent width in hei_8446 MaskedColumn 39455
      eqw_heii_1640 float64                                                           Observed-frame equivalent width in heii_1640 MaskedColumn 61666
      eqw_heii_4687 float64                                                           Observed-frame equivalent width in heii_4687 MaskedColumn 59316
             eqw_hg float64                                                                  Observed-frame equivalent width in hg MaskedColumn 38924
            eqw_lya float64                                                                 Observed-frame equivalent width in lya MaskedColumn 68665
           eqw_mgii float64                                                                Observed-frame equivalent width in mgii MaskedColumn 49348
     eqw_neiii_3867 float64                                                          Observed-frame equivalent width in neiii_3867 MaskedColumn 61873
     eqw_neiii_3968 float64                                                          Observed-frame equivalent width in neiii_3968 MaskedColumn 41256
       eqw_nev_3346 float64                                                            Observed-frame equivalent width in nev_3346 MaskedColumn 45515
      eqw_nevi_3426 float64                                                           Observed-frame equivalent width in nevi_3426 MaskedColumn 44964
      eqw_niii_1750 float64                                                           Observed-frame equivalent width in niii_1750 MaskedColumn 59948
       eqw_niv_1487 float64                                                            Observed-frame equivalent width in niv_1487 MaskedColumn 65764
        eqw_oi_6302 float64                                                             Observed-frame equivalent width in oi_6302 MaskedColumn 34357
            eqw_oii float64                                                                 Observed-frame equivalent width in oii MaskedColumn 42852
       eqw_oii_7325 float64                                                            Observed-frame equivalent width in oii_7325 MaskedColumn 50584
           eqw_oiii float64                                                                Observed-frame equivalent width in oiii MaskedColumn 57734
      eqw_oiii_1663 float64                                                           Observed-frame equivalent width in oiii_1663 MaskedColumn 61065
      eqw_oiii_4363 float64                                                           Observed-frame equivalent width in oiii_4363 MaskedColumn 60240
      eqw_oiii_4959 float64                                                           Observed-frame equivalent width in oiii_4959 MaskedColumn 58736
      eqw_oiii_5007 float64                                                           Observed-frame equivalent width in oiii_5007 MaskedColumn 58608
           eqw_pa10 float64                                                                Observed-frame equivalent width in pa10 MaskedColumn 41179
            eqw_pa8 float64                                                                 Observed-frame equivalent width in pa8 MaskedColumn 42725
            eqw_pa9 float64                                                                 Observed-frame equivalent width in pa9 MaskedColumn 41798
            eqw_paa float64                                                                 Observed-frame equivalent width in paa MaskedColumn 63671
            eqw_pab float64                                                                 Observed-frame equivalent width in pab MaskedColumn 52477
            eqw_pad float64                                                                 Observed-frame equivalent width in pad MaskedColumn 44239
            eqw_pag float64                                                                 Observed-frame equivalent width in pag MaskedColumn 46592
            eqw_pfb float64                                                                 Observed-frame equivalent width in pfb MaskedColumn 79212
            eqw_pfd float64                                                                 Observed-frame equivalent width in pfd MaskedColumn 74920
            eqw_pfe float64                                                                 Observed-frame equivalent width in pfe MaskedColumn 73680
            eqw_pfg float64                                                                 Observed-frame equivalent width in pfg MaskedColumn 76556
            eqw_sii float64                                                                 Observed-frame equivalent width in sii MaskedColumn 49501
      eqw_siii_9068 float64                                                           Observed-frame equivalent width in siii_9068 MaskedColumn 41361
      eqw_siii_9531 float64                                                           Observed-frame equivalent width in siii_9531 MaskedColumn 42685
        line_ha_nii float64                                                            Line flux of combined Ha+NII with 3:1 ratio MaskedColumn 49269
    line_ha_nii_err float64                                                                                                        MaskedColumn 49273
            eqw_h10 float64                                                                 Observed-frame equivalent width in h10 MaskedColumn 71081
            eqw_h11 float64                                                                 Observed-frame equivalent width in h11 MaskedColumn 71209
            eqw_h12 float64                                                                 Observed-frame equivalent width in h12 MaskedColumn 71254
             eqw_h7 float64                                                                  Observed-frame equivalent width in h7 MaskedColumn 70536
             eqw_h8 float64                                                                  Observed-frame equivalent width in h8 MaskedColumn 70806
             eqw_h9 float64                                                                  Observed-frame equivalent width in h9 MaskedColumn 70958
             eqw_ha float64                                                                  Observed-frame equivalent width in ha MaskedColumn 65643
       eqw_hei_6680 float64                                                            Observed-frame equivalent width in hei_6680 MaskedColumn 65749
       eqw_nii_6549 float64                                                            Observed-frame equivalent width in nii_6549 MaskedColumn 65635
       eqw_nii_6584 float64                                                            Observed-frame equivalent width in nii_6584 MaskedColumn 65647
       eqw_oii_7323 float64                                                            Observed-frame equivalent width in oii_7323 MaskedColumn 66230
       eqw_oii_7332 float64                                                            Observed-frame equivalent width in oii_7332 MaskedColumn 66244
       eqw_sii_6717 float64                                                            Observed-frame equivalent width in sii_6717 MaskedColumn 65776
       eqw_sii_6731 float64                                                            Observed-frame equivalent width in sii_6731 MaskedColumn 65760
      eqw_siii_6314 float64                                                           Observed-frame equivalent width in siii_6314 MaskedColumn 65692
            sn_line float64           .1f                                                                 Maximum emission line SN MaskedColumn 18242
              ztime float64                                                                              UNIX time of redshift fit MaskedColumn 18242
       line_ci_9850 float64                                                                   Line flux of ci_9850 1e-20 erg/s/cm2 MaskedColumn 43554
   line_ci_9850_err float64                                                                                                        MaskedColumn 43558
    line_feii_11128 float64                                                                Line flux of feii_11128 1e-20 erg/s/cm2 MaskedColumn 47164
line_feii_11128_err float64                                                                                                        MaskedColumn 47167
     line_pii_11886 float64                                                                 Line flux of pii_11886 1e-20 erg/s/cm2 MaskedColumn 49532
 line_pii_11886_err float64                                                                                                        MaskedColumn 49532
    line_feii_12570 float64                                                                Line flux of feii_12570 1e-20 erg/s/cm2 MaskedColumn 51790
line_feii_12570_err float64                                                                                                        MaskedColumn 51790
        eqw_ci_9850 float64                                                             Observed-frame equivalent width in ci_9850 MaskedColumn 43588
     eqw_feii_11128 float64                                                          Observed-frame equivalent width in feii_11128 MaskedColumn 47206
      eqw_pii_11886 float64                                                           Observed-frame equivalent width in pii_11886 MaskedColumn 49563
     eqw_feii_12570 float64                                                          Observed-frame equivalent width in feii_12570 MaskedColumn 51827
    line_feii_16440 float64                                                                Line flux of feii_16440 1e-20 erg/s/cm2 MaskedColumn 59928
line_feii_16440_err float64                                                                                                        MaskedColumn 59928
    line_feii_16877 float64                                                                Line flux of feii_16877 1e-20 erg/s/cm2 MaskedColumn 60978
line_feii_16877_err float64                                                                                                        MaskedColumn 60978
           line_brf float64                                                                       Line flux of brf 1e-20 erg/s/cm2 MaskedColumn 61730
       line_brf_err float64                                                                                                        MaskedColumn 61730
    line_feii_17418 float64                                                                Line flux of feii_17418 1e-20 erg/s/cm2 MaskedColumn 61813
line_feii_17418_err float64                                                                                                        MaskedColumn 61813
           line_bre float64                                                                       Line flux of bre 1e-20 erg/s/cm2 MaskedColumn 62881
       line_bre_err float64                                                                                                        MaskedColumn 62882
    line_feii_18362 float64                                                                Line flux of feii_18362 1e-20 erg/s/cm2 MaskedColumn 63211
line_feii_18362_err float64                                                                                                        MaskedColumn 63211
     eqw_feii_16440 float64                                                          Observed-frame equivalent width in feii_16440 MaskedColumn 59946
     eqw_feii_16877 float64                                                          Observed-frame equivalent width in feii_16877 MaskedColumn 60995
            eqw_brf float64                                                                 Observed-frame equivalent width in brf MaskedColumn 61749
     eqw_feii_17418 float64                                                          Observed-frame equivalent width in feii_17418 MaskedColumn 61829
            eqw_bre float64                                                                 Observed-frame equivalent width in bre MaskedColumn 62895
     eqw_feii_18362 float64                                                          Observed-frame equivalent width in feii_18362 MaskedColumn 63227
              valid    str5                                                         Redshift matches best z from visual inspection MaskedColumn 38983
              objid   int64                                                                               Unique source identifier       Column     0
             z_best float64                                                              Best redshift estimate for unique sources       Column     0
              ztype    str1                                                                 Source for z_best (G)rating or (P)rism MaskedColumn 38728
            z_prism float64                                                              Best redshift estimate from prism spectra       Column     0
          z_grating float64                                                            Best redshift estiamte from grating spectra       Column     0
    phot_correction float64           .2f Scale to photometry -log10(c) = -0.902 log10(flux_radius) + 0.649 log10(profsig) + 0.605 MaskedColumn 29922
   phot_flux_radius float64           .2f                                                      FLUX_RADIUS from photometric source MaskedColumn 29922
            phot_dr float64                                      Offset to the nearest source in the photometric catalog in arcsec MaskedColumn 29922
          file_phot   str44                                                                    Filename of the photometric catalog MaskedColumn 29922
            id_phot   int64                                                                  ID number in the photometric cadtalog MaskedColumn 29922
      phot_mag_auto float64           .2f                                         Kron MAG_AUTO in the photometric detection image MaskedColumn 29922
   phot_f090w_tot_1 float64                                                  Total f090w flux density from the photometric catalog MaskedColumn 29922
  phot_f090w_etot_1 float64                                                                                                        MaskedColumn 29922
   phot_f115w_tot_1 float64                                                  Total f115w flux density from the photometric catalog MaskedColumn 29922
  phot_f115w_etot_1 float64                                                                                                        MaskedColumn 29922
   phot_f150w_tot_1 float64                                                  Total f150w flux density from the photometric catalog MaskedColumn 29922
  phot_f150w_etot_1 float64                                                                                                        MaskedColumn 29922
   phot_f200w_tot_1 float64                                                  Total f200w flux density from the photometric catalog MaskedColumn 29922
  phot_f200w_etot_1 float64                                                                                                        MaskedColumn 29922
   phot_f277w_tot_1 float64                                                  Total f277w flux density from the photometric catalog MaskedColumn 29922
  phot_f277w_etot_1 float64                                                                                                        MaskedColumn 29922
   phot_f356w_tot_1 float64                                                  Total f356w flux density from the photometric catalog MaskedColumn 29922
  phot_f356w_etot_1 float64                                                                                                        MaskedColumn 29922
   phot_f410m_tot_1 float64                                                  Total f410m flux density from the photometric catalog MaskedColumn 29922
  phot_f410m_etot_1 float64                                                                                                        MaskedColumn 29922
   phot_f444w_tot_1 float64                                                  Total f444w flux density from the photometric catalog MaskedColumn 29922
  phot_f444w_etot_1 float64                                                                                                        MaskedColumn 29922
            phot_Av float64                                                                                 Av from the photoz fit MaskedColumn 29922
          phot_mass float64                                                                       stellar mass from the photoz fit MaskedColumn 29922
         phot_restU float64                                              flux density of the redshifted U band from the photoz fit MaskedColumn 29922
         phot_restV float64                                              flux density of the redshifted V band from the photoz fit MaskedColumn 29922
         phot_restJ float64                                              flux density of the redshifted J band from the photoz fit MaskedColumn 29922
             z_phot float64                                                                                   photometric redshift MaskedColumn 29922
           phot_LHa float64                                                                 Halpha luminosity from the photo-z fit MaskedColumn 29922
         phot_LOIII float64                                                                   OIII luminosity from the photo-z fit MaskedColumn 29922
          phot_LOII float64                                                                    OII luminosity from the photo-z fit MaskedColumn 29922
              grade   int64                                                                           Grade from visual inspection MaskedColumn 38015
             zgrade float64          9.5f                                                          Redshift from visual inspection MaskedColumn 38015
           reviewer    str4                                                             Initials of the visual inspection reviewer MaskedColumn 38015
            comment   str93                                                                         Comment from visual inspection MaskedColumn 48371
                zrf float64                                                         Redshift used for integrated rest-frame filter MaskedColumn 18242
             escale float64                                                                                                        MaskedColumn 18242
      obs_239_valid   int64                                                                                                        MaskedColumn 18242
       obs_239_frac float64                                                      Fraction of wfc_f814w_t81.dat covered by spectrum MaskedColumn 18242
       obs_239_flux float64                                                                     Spectrum flux in wfc_f814w_t81.dat MaskedColumn 18242
        obs_239_err float64                                                                      Spectrum err in wfc_f814w_t81.dat MaskedColumn 18242
   obs_239_full_err float64                                                                      Spectrum err in wfc_f814w_t81.dat MaskedColumn 18242
      obs_205_valid   int64                                                                                                        MaskedColumn 18242
       obs_205_frac float64                                                              Fraction of f160w.dat covered by spectrum MaskedColumn 18242
       obs_205_flux float64                                                                             Spectrum flux in f160w.dat MaskedColumn 18242
        obs_205_err float64                                                                              Spectrum err in f160w.dat MaskedColumn 18242
   obs_205_full_err float64                                                                              Spectrum err in f160w.dat MaskedColumn 18242
      obs_362_valid   int64                                                                                                        MaskedColumn 18242
       obs_362_frac float64                                                      Fraction of jwst_nircam_f070w covered by spectrum MaskedColumn 18242
       obs_362_flux float64                                                                     Spectrum flux in jwst_nircam_f070w MaskedColumn 18242
        obs_362_err float64                                                                      Spectrum err in jwst_nircam_f070w MaskedColumn 18242
   obs_362_full_err float64                                                                      Spectrum err in jwst_nircam_f070w MaskedColumn 18242
      obs_363_valid   int64                                                                                                        MaskedColumn 18242
       obs_363_frac float64                                                      Fraction of jwst_nircam_f090w covered by spectrum MaskedColumn 18242
       obs_363_flux float64                                                                     Spectrum flux in jwst_nircam_f090w MaskedColumn 18242
        obs_363_err float64                                                                      Spectrum err in jwst_nircam_f090w MaskedColumn 18242
   obs_363_full_err float64                                                                      Spectrum err in jwst_nircam_f090w MaskedColumn 18242
      obs_364_valid   int64                                                                                                        MaskedColumn 18242
       obs_364_frac float64                                                      Fraction of jwst_nircam_f115w covered by spectrum MaskedColumn 18242
       obs_364_flux float64                                                                     Spectrum flux in jwst_nircam_f115w MaskedColumn 18242
        obs_364_err float64                                                                      Spectrum err in jwst_nircam_f115w MaskedColumn 18242
   obs_364_full_err float64                                                                      Spectrum err in jwst_nircam_f115w MaskedColumn 18242
      obs_365_valid   int64                                                                                                        MaskedColumn 18242
       obs_365_frac float64                                                      Fraction of jwst_nircam_f150w covered by spectrum MaskedColumn 18242
       obs_365_flux float64                                                                     Spectrum flux in jwst_nircam_f150w MaskedColumn 18242
        obs_365_err float64                                                                      Spectrum err in jwst_nircam_f150w MaskedColumn 18242
   obs_365_full_err float64                                                                      Spectrum err in jwst_nircam_f150w MaskedColumn 18242
      obs_366_valid   int64                                                                                                        MaskedColumn 18242
       obs_366_frac float64                                                      Fraction of jwst_nircam_f200w covered by spectrum MaskedColumn 18242
       obs_366_flux float64                                                                     Spectrum flux in jwst_nircam_f200w MaskedColumn 18242
        obs_366_err float64                                                                      Spectrum err in jwst_nircam_f200w MaskedColumn 18242
   obs_366_full_err float64                                                                      Spectrum err in jwst_nircam_f200w MaskedColumn 18242
      obs_370_valid   int64                                                                                                        MaskedColumn 18242
       obs_370_frac float64                                                      Fraction of jwst_nircam_f182m covered by spectrum MaskedColumn 18242
       obs_370_flux float64                                                                     Spectrum flux in jwst_nircam_f182m MaskedColumn 18242
        obs_370_err float64                                                                      Spectrum err in jwst_nircam_f182m MaskedColumn 18242
   obs_370_full_err float64                                                                      Spectrum err in jwst_nircam_f182m MaskedColumn 18242
      obs_371_valid   int64                                                                                                        MaskedColumn 18242
       obs_371_frac float64                                                      Fraction of jwst_nircam_f210m covered by spectrum MaskedColumn 18242
       obs_371_flux float64                                                                     Spectrum flux in jwst_nircam_f210m MaskedColumn 18242
        obs_371_err float64                                                                      Spectrum err in jwst_nircam_f210m MaskedColumn 18242
   obs_371_full_err float64                                                                      Spectrum err in jwst_nircam_f210m MaskedColumn 18242
      obs_375_valid   int64                                                                                                        MaskedColumn 18242
       obs_375_frac float64                                                      Fraction of jwst_nircam_f277w covered by spectrum MaskedColumn 18242
       obs_375_flux float64                                                                     Spectrum flux in jwst_nircam_f277w MaskedColumn 18242
        obs_375_err float64                                                                      Spectrum err in jwst_nircam_f277w MaskedColumn 18242
   obs_375_full_err float64                                                                      Spectrum err in jwst_nircam_f277w MaskedColumn 18242
      obs_376_valid   int64                                                                                                        MaskedColumn 18242
       obs_376_frac float64                                                      Fraction of jwst_nircam_f356w covered by spectrum MaskedColumn 18242
       obs_376_flux float64                                                                     Spectrum flux in jwst_nircam_f356w MaskedColumn 18242
        obs_376_err float64                                                                      Spectrum err in jwst_nircam_f356w MaskedColumn 18242
   obs_376_full_err float64                                                                      Spectrum err in jwst_nircam_f356w MaskedColumn 18242
      obs_377_valid   int64                                                                                                        MaskedColumn 18242
       obs_377_frac float64                                                      Fraction of jwst_nircam_f444w covered by spectrum MaskedColumn 18242
       obs_377_flux float64                                                                     Spectrum flux in jwst_nircam_f444w MaskedColumn 18242
        obs_377_err float64                                                                      Spectrum err in jwst_nircam_f444w MaskedColumn 18242
   obs_377_full_err float64                                                                      Spectrum err in jwst_nircam_f444w MaskedColumn 18242
      obs_379_valid   int64                                                                                                        MaskedColumn 18242
       obs_379_frac float64                                                      Fraction of jwst_nircam_f250m covered by spectrum MaskedColumn 18242
       obs_379_flux float64                                                                     Spectrum flux in jwst_nircam_f250m MaskedColumn 18242
        obs_379_err float64                                                                      Spectrum err in jwst_nircam_f250m MaskedColumn 18242
   obs_379_full_err float64                                                                      Spectrum err in jwst_nircam_f250m MaskedColumn 18242
      obs_380_valid   int64                                                                                                        MaskedColumn 18242
       obs_380_frac float64                                                      Fraction of jwst_nircam_f300m covered by spectrum MaskedColumn 18242
       obs_380_flux float64                                                                     Spectrum flux in jwst_nircam_f300m MaskedColumn 18242
        obs_380_err float64                                                                      Spectrum err in jwst_nircam_f300m MaskedColumn 18242
   obs_380_full_err float64                                                                      Spectrum err in jwst_nircam_f300m MaskedColumn 18242
      obs_381_valid   int64                                                                                                        MaskedColumn 18242
       obs_381_frac float64                                                      Fraction of jwst_nircam_f335m covered by spectrum MaskedColumn 18242
       obs_381_flux float64                                                                     Spectrum flux in jwst_nircam_f335m MaskedColumn 18242
        obs_381_err float64                                                                      Spectrum err in jwst_nircam_f335m MaskedColumn 18242
   obs_381_full_err float64                                                                      Spectrum err in jwst_nircam_f335m MaskedColumn 18242
      obs_382_valid   int64                                                                                                        MaskedColumn 18242
       obs_382_frac float64                                                      Fraction of jwst_nircam_f360m covered by spectrum MaskedColumn 18242
       obs_382_flux float64                                                                     Spectrum flux in jwst_nircam_f360m MaskedColumn 18242
        obs_382_err float64                                                                      Spectrum err in jwst_nircam_f360m MaskedColumn 18242
   obs_382_full_err float64                                                                      Spectrum err in jwst_nircam_f360m MaskedColumn 18242
      obs_383_valid   int64                                                                                                        MaskedColumn 18242
       obs_383_frac float64                                                      Fraction of jwst_nircam_f410m covered by spectrum MaskedColumn 18242
       obs_383_flux float64                                                                     Spectrum flux in jwst_nircam_f410m MaskedColumn 18242
        obs_383_err float64                                                                      Spectrum err in jwst_nircam_f410m MaskedColumn 18242
   obs_383_full_err float64                                                                      Spectrum err in jwst_nircam_f410m MaskedColumn 18242
      obs_384_valid   int64                                                                                                        MaskedColumn 18242
       obs_384_frac float64                                                      Fraction of jwst_nircam_f430m covered by spectrum MaskedColumn 18242
       obs_384_flux float64                                                                     Spectrum flux in jwst_nircam_f430m MaskedColumn 18242
        obs_384_err float64                                                                      Spectrum err in jwst_nircam_f430m MaskedColumn 18242
   obs_384_full_err float64                                                                      Spectrum err in jwst_nircam_f430m MaskedColumn 18242
      obs_385_valid   int64                                                                                                        MaskedColumn 18242
       obs_385_frac float64                                                      Fraction of jwst_nircam_f460m covered by spectrum MaskedColumn 18242
       obs_385_flux float64                                                                     Spectrum flux in jwst_nircam_f460m MaskedColumn 18242
        obs_385_err float64                                                                      Spectrum err in jwst_nircam_f460m MaskedColumn 18242
   obs_385_full_err float64                                                                      Spectrum err in jwst_nircam_f460m MaskedColumn 18242
      obs_386_valid   int64                                                                                                        MaskedColumn 18242
       obs_386_frac float64                                                      Fraction of jwst_nircam_f480m covered by spectrum MaskedColumn 18242
       obs_386_flux float64                                                                     Spectrum flux in jwst_nircam_f480m MaskedColumn 18242
        obs_386_err float64                                                                      Spectrum err in jwst_nircam_f480m MaskedColumn 18242
   obs_386_full_err float64                                                                      Spectrum err in jwst_nircam_f480m MaskedColumn 18242
     rest_120_valid   int64                                                                                                        MaskedColumn 18242
      rest_120_frac float64                                                          Fraction of galex1500.res covered by spectrum MaskedColumn 18242
      rest_120_flux float64                                                                         Spectrum flux in galex1500.res MaskedColumn 18242
       rest_120_err float64                                                                          Spectrum err in galex1500.res MaskedColumn 18242
  rest_120_full_err float64                                                                          Spectrum err in galex1500.res MaskedColumn 18242
     rest_121_valid   int64                                                                                                        MaskedColumn 18242
      rest_121_frac float64                                                          Fraction of galex2500.res covered by spectrum MaskedColumn 18242
      rest_121_flux float64                                                                         Spectrum flux in galex2500.res MaskedColumn 18242
       rest_121_err float64                                                                          Spectrum err in galex2500.res MaskedColumn 18242
  rest_121_full_err float64                                                                          Spectrum err in galex2500.res MaskedColumn 18242
     rest_218_valid   int64                                                                                                        MaskedColumn 18242
      rest_218_frac float64                                                             Fraction of UV1600.dat covered by spectrum MaskedColumn 18242
      rest_218_flux float64                                                                            Spectrum flux in UV1600.dat MaskedColumn 18242
       rest_218_err float64                                                                             Spectrum err in UV1600.dat MaskedColumn 18242
  rest_218_full_err float64                                                                             Spectrum err in UV1600.dat MaskedColumn 18242
     rest_219_valid   int64                                                                                                        MaskedColumn 18242
      rest_219_frac float64                                                             Fraction of UV2800.dat covered by spectrum MaskedColumn 18242
      rest_219_flux float64                                                                            Spectrum flux in UV2800.dat MaskedColumn 18242
       rest_219_err float64                                                                             Spectrum err in UV2800.dat MaskedColumn 18242
  rest_219_full_err float64                                                                             Spectrum err in UV2800.dat MaskedColumn 18242
     rest_270_valid   int64                                                                                                        MaskedColumn 18242
      rest_270_frac float64                                                    Fraction of Tophat_1400_200.dat covered by spectrum MaskedColumn 18242
      rest_270_flux float64                                                                   Spectrum flux in Tophat_1400_200.dat MaskedColumn 18242
       rest_270_err float64                                                                    Spectrum err in Tophat_1400_200.dat MaskedColumn 18242
  rest_270_full_err float64                                                                    Spectrum err in Tophat_1400_200.dat MaskedColumn 18242
     rest_271_valid   int64                                                                                                        MaskedColumn 18242
      rest_271_frac float64                                                    Fraction of Tophat_1700_200.dat covered by spectrum MaskedColumn 18242
      rest_271_flux float64                                                                   Spectrum flux in Tophat_1700_200.dat MaskedColumn 18242
       rest_271_err float64                                                                    Spectrum err in Tophat_1700_200.dat MaskedColumn 18242
  rest_271_full_err float64                                                                    Spectrum err in Tophat_1700_200.dat MaskedColumn 18242
     rest_272_valid   int64                                                                                                        MaskedColumn 18242
      rest_272_frac float64                                                    Fraction of Tophat_2200_200.dat covered by spectrum MaskedColumn 18242
      rest_272_flux float64                                                                   Spectrum flux in Tophat_2200_200.dat MaskedColumn 18242
       rest_272_err float64                                                                    Spectrum err in Tophat_2200_200.dat MaskedColumn 18242
  rest_272_full_err float64                                                                    Spectrum err in Tophat_2200_200.dat MaskedColumn 18242
     rest_274_valid   int64                                                                                                        MaskedColumn 18242
      rest_274_frac float64                                                    Fraction of Tophat_2800_200.dat covered by spectrum MaskedColumn 18242
      rest_274_flux float64                                                                   Spectrum flux in Tophat_2800_200.dat MaskedColumn 18242
       rest_274_err float64                                                                    Spectrum err in Tophat_2800_200.dat MaskedColumn 18242
  rest_274_full_err float64                                                                    Spectrum err in Tophat_2800_200.dat MaskedColumn 18242
     rest_153_valid   int64                                                                                                        MaskedColumn 18242
      rest_153_frac float64                                           Fraction of maiz-apellaniz_Johnson_U.res covered by spectrum MaskedColumn 18242
      rest_153_flux float64                                                          Spectrum flux in maiz-apellaniz_Johnson_U.res MaskedColumn 18242
       rest_153_err float64                                                           Spectrum err in maiz-apellaniz_Johnson_U.res MaskedColumn 18242
  rest_153_full_err float64                                                           Spectrum err in maiz-apellaniz_Johnson_U.res MaskedColumn 18242
     rest_154_valid   int64                                                                                                        MaskedColumn 18242
      rest_154_frac float64                                           Fraction of maiz-apellaniz_Johnson_B.res covered by spectrum MaskedColumn 18242
      rest_154_flux float64                                                          Spectrum flux in maiz-apellaniz_Johnson_B.res MaskedColumn 18242
       rest_154_err float64                                                           Spectrum err in maiz-apellaniz_Johnson_B.res MaskedColumn 18242
  rest_154_full_err float64                                                           Spectrum err in maiz-apellaniz_Johnson_B.res MaskedColumn 18242
     rest_155_valid   int64                                                                                                        MaskedColumn 18242
      rest_155_frac float64                                           Fraction of maiz-apellaniz_Johnson_V.res covered by spectrum MaskedColumn 18242
      rest_155_flux float64                                                          Spectrum flux in maiz-apellaniz_Johnson_V.res MaskedColumn 18242
       rest_155_err float64                                                           Spectrum err in maiz-apellaniz_Johnson_V.res MaskedColumn 18242
  rest_155_full_err float64                                                           Spectrum err in maiz-apellaniz_Johnson_V.res MaskedColumn 18242
     rest_156_valid   int64                                                                                                        MaskedColumn 18242
      rest_156_frac float64                                                                  Fraction of u.dat covered by spectrum MaskedColumn 18242
      rest_156_flux float64                                                                                 Spectrum flux in u.dat MaskedColumn 18242
       rest_156_err float64                                                                                  Spectrum err in u.dat MaskedColumn 18242
  rest_156_full_err float64                                                                                  Spectrum err in u.dat MaskedColumn 18242
     rest_157_valid   int64                                                                                                        MaskedColumn 18242
      rest_157_frac float64                                                                  Fraction of g.dat covered by spectrum MaskedColumn 18242
      rest_157_flux float64                                                                                 Spectrum flux in g.dat MaskedColumn 18242
       rest_157_err float64                                                                                  Spectrum err in g.dat MaskedColumn 18242
  rest_157_full_err float64                                                                                  Spectrum err in g.dat MaskedColumn 18242
     rest_158_valid   int64                                                                                                        MaskedColumn 18242
      rest_158_frac float64                                                                  Fraction of r.dat covered by spectrum MaskedColumn 18242
      rest_158_flux float64                                                                                 Spectrum flux in r.dat MaskedColumn 18242
       rest_158_err float64                                                                                  Spectrum err in r.dat MaskedColumn 18242
  rest_158_full_err float64                                                                                  Spectrum err in r.dat MaskedColumn 18242
     rest_159_valid   int64                                                                                                        MaskedColumn 18242
      rest_159_frac float64                                                                  Fraction of i.dat covered by spectrum MaskedColumn 18242
      rest_159_flux float64                                                                                 Spectrum flux in i.dat MaskedColumn 18242
       rest_159_err float64                                                                                  Spectrum err in i.dat MaskedColumn 18242
  rest_159_full_err float64                                                                                  Spectrum err in i.dat MaskedColumn 18242
     rest_160_valid   int64                                                                                                        MaskedColumn 18242
      rest_160_frac float64                                                                  Fraction of z.dat covered by spectrum MaskedColumn 18242
      rest_160_flux float64                                                                                 Spectrum flux in z.dat MaskedColumn 18242
       rest_160_err float64                                                                                  Spectrum err in z.dat MaskedColumn 18242
  rest_160_full_err float64                                                                                  Spectrum err in z.dat MaskedColumn 18242
     rest_161_valid   int64                                                                                                        MaskedColumn 18242
      rest_161_frac float64                                                                  Fraction of J.res covered by spectrum MaskedColumn 18242
      rest_161_flux float64                                                                                 Spectrum flux in J.res MaskedColumn 18242
       rest_161_err float64                                                                                  Spectrum err in J.res MaskedColumn 18242
  rest_161_full_err float64                                                                                  Spectrum err in J.res MaskedColumn 18242
     rest_162_valid   int64                                                                                                        MaskedColumn 18242
      rest_162_frac float64                                                                  Fraction of H.res covered by spectrum MaskedColumn 18242
      rest_162_flux float64                                                                                 Spectrum flux in H.res MaskedColumn 18242
       rest_162_err float64                                                                                  Spectrum err in H.res MaskedColumn 18242
  rest_162_full_err float64                                                                                  Spectrum err in H.res MaskedColumn 18242
     rest_163_valid   int64                                                                                                        MaskedColumn 18242
      rest_163_frac float64                                                                  Fraction of K.res covered by spectrum MaskedColumn 18242
      rest_163_flux float64                                                                                 Spectrum flux in K.res MaskedColumn 18242
       rest_163_err float64                                                                                  Spectrum err in K.res MaskedColumn 18242
  rest_163_full_err float64                                                                                  Spectrum err in K.res MaskedColumn 18242
     rest_414_valid   int64                                                                                                        MaskedColumn 18242
      rest_414_frac float64                                                            Fraction of synthetic_u covered by spectrum MaskedColumn 18242
      rest_414_flux float64                                                                           Spectrum flux in synthetic_u MaskedColumn 18242
       rest_414_err float64                                                                            Spectrum err in synthetic_u MaskedColumn 18242
  rest_414_full_err float64                                                                            Spectrum err in synthetic_u MaskedColumn 18242
     rest_415_valid   int64                                                                                                        MaskedColumn 18242
      rest_415_frac float64                                                            Fraction of synthetic_g covered by spectrum MaskedColumn 18242
      rest_415_flux float64                                                                           Spectrum flux in synthetic_g MaskedColumn 18242
       rest_415_err float64                                                                            Spectrum err in synthetic_g MaskedColumn 18242
  rest_415_full_err float64                                                                            Spectrum err in synthetic_g MaskedColumn 18242
     rest_416_valid   int64                                                                                                        MaskedColumn 18242
      rest_416_frac float64                                                            Fraction of synthetic_i covered by spectrum MaskedColumn 18242
      rest_416_flux float64                                                                           Spectrum flux in synthetic_i MaskedColumn 18242
       rest_416_err float64                                                                            Spectrum err in synthetic_i MaskedColumn 18242
  rest_416_full_err float64                                                                            Spectrum err in synthetic_i MaskedColumn 18242
               beta float64                                                                                Estimated UV slope beta MaskedColumn 51222
      beta_ref_flux float64                                                                                                        MaskedColumn 51221
          beta_npix   int64                                                                          Number of pixels for beta fit MaskedColumn 18242
           beta_wlo float64                                                                   Minimum wavelength used for beta fit MaskedColumn 51221
           beta_whi float64                                                                   Maximum wavelength used for beta fit MaskedColumn 51221
          beta_nmad float64                                                                                   NMAD of the beta fit MaskedColumn 51225
           dla_npix float64                                                                       Number of pixels for the DLA fit MaskedColumn 51221
          dla_value float64                                                                       DLA parameter from Heintz et al. MaskedColumn 51221
            dla_unc float64                                                                           Uncertainty on DLA parameter MaskedColumn 51221
        beta_cov_00 float64                                                               Components of the beta covariance matrix MaskedColumn 51221
        beta_cov_01 float64                                                               Components of the beta covariance matrix MaskedColumn 51221
        beta_cov_10 float64                                                               Components of the beta covariance matrix MaskedColumn 51221
        beta_cov_11 float64                                                               Components of the beta covariance matrix MaskedColumn 51221

Add some preview columns to the table

RGB_URL = "https://grizli-cutout.herokuapp.com/thumb?size=1.5&scl=2.0&asinh=True&filters=f115w-clear%2Cf277w-clear%2Cf444w-clear&rgb_scl=1.5%2C0.74%2C1.3&pl=2&coord={ra}%2C{dec}"
tab['metafile'] = [m.split('_')[0] for m in tab['msamet']]
SLIT_URL = "https://grizli-cutout.herokuapp.com/thumb?size=1.5&scl=4.0&invert=True&filters=f444w-clear&rgb_scl=1.5%2C0.74%2C1.3&pl=2&coord={ra}%2C{dec}&nirspec=True&dpi_scale=6&nrs_lw=0.5&nrs_alpha=0.8&metafile={metafile}"
FITS_URL = "https://s3.amazonaws.com/msaexp-nirspec/extractions/{root}/{file}"

tab['Thumb'] = [
    "<img src=\"{0}\" height=200px>".format(
        RGB_URL.format(**row['ra','dec'])
    )
    for row in tab
]

tab['Slit_Thumb'] = [
    "<img src=\"{0}\" height=200px>".format(
        SLIT_URL.format(**row['ra','dec','metafile'])
    )
    for row in tab
]

tab['Spectrum_fnu'] = [
    "<img src=\"{0}\" height=200px>".format(
        FITS_URL.format(**row['root','file']).replace('.spec.fits', '.fnu.png')
    )
    for row in tab
]

tab['Spectrum_flam'] = [
    "<img src=\"{0}\" height=200px>".format(
        FITS_URL.format(**row['root','file']).replace('.spec.fits', '.flam.png')
    )
    for row in tab
]

# CANUCS in a different bucket
canucs = np.isin(
    tab['root'],
    ['abell370-v4', 'macs0416-v4', 'macs0417-v4', 'macs1149-v4', 'macs1423-v4']
)

for j in tqdm(np.where(canucs)[0]):
    for c in ['Spectrum_fnu', 'Spectrum_flam']:
        tab[c][j] = tab[c][j].replace(
            'msaexp-nirspec/extractions',
            'grizli-canucs/nirspec'
        )

100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 1534/1534 [00:00<00:00, 97100.33it/s]

zphot - zspec

Compare the “best” NIRSpec redshift with grade=3 (grating if available, prism otherwise) to the photometric redshift in the matched catalogs.

import eazy.utils
test = (tab['grade'] == 3) & (tab['z_phot'].filled(-1.) > 0)
test &= (tab['grating'] == 'PRISM')
print(test.sum())
_ = eazy.utils.zphot_zspec(tab['z_phot'][test], tab['z_best'][test], zmax=14)


13703

png

# Counts by mask / program
utils.Unique(tab['root'], sort_counts=False)

   N  value     
====  ==========
 314  abell2744-castellano1-v4
 613  abell2744-castellano2-v4
 114  abell2744-ddt-v4
 462  abell2744-glass-v4
 299  abell370-v4
 141  aurora-gdn01-v4
 153  aurora-gdn02-v4
   9  bd-ic348-gto-v4
  25  bd-orion-gto-v4
 344  bd-orion-gto2-v4
 219  bluejay-north-v4
 224  bluejay-south-v4
  52  borg-0037m3337-v4
  39  borg-0314m6712-v4
  37  borg-0409m5317-v4
  36  borg-0440m5244-v4
  38  borg-0859p4114-v4
  48  borg-0955p4528-v4
  38  borg-1033p5051-v4
  43  borg-1437p5044-v4
  31  borg-2203p1851-v4
1107  cal-m31-pn2538-v4
 414  cal-ocen-degraaff-v4
  63  cantalupo-filament-02-v4
 338  capers-cos01-v4
 286  capers-cos04-v4
 377  capers-cos07-v4
 417  capers-cos10-v4
 292  capers-cos13-v4
 306  capers-cos16-v4
 377  capers-cos19-v4
 376  capers-egs44-v4
 396  capers-egs47-v4
 408  capers-egs49-v4
 415  capers-egs53-v4
 328  capers-egs55-v4
 389  capers-egs61-v4
 372  capers-egs65-v4
 351  capers-udsp1-v4
 302  capers-udsp2-v4
 281  capers-udsp3-v4
 170  capers-udsp5-v4
 102  cecilia-v4
 259  ceers-ddt-v4
1818  ceers-v4  
  96  cosmos-alpha-v4
 367  cosmos-curti-v4
  71  cosmos-lae-martin-v4
 307  cosmos-transients-v4
  84  cristal-cos01-v4
 342  egs-mason-v4
 272  egs-nelsonx-v4
 127  excels-uds01-v4
 128  excels-uds02-v4
 136  excels-uds03-v4
 142  excels-uds04-v4
  44  gdn-chisholm-v4
 580  gdn-fujimoto-v4
 190  gdn-pah123-v4
  66  gdn-pah4-v4
  57  gds-barrufet-s156-v4
  82  gds-barrufet-s67-v4
1236  gds-deep-v4
 356  gds-egami-ddt-v4
 256  gds-maseda-v4
 724  gds-rieke-v4
 825  gds-udeep-v4
  96  glazebrook-cos-obs1-v4
 110  glazebrook-cos-obs2-v4
 108  glazebrook-cos-obs3-v4
 122  glazebrook-egs-v4
 238  glazebrook-v4
 129  glimpse-obs01-v4
 118  glimpse-obs01b-v4
 135  glimpse-obs02-v4
 192  goodsn-wide-v4
 594  goodsn-wide0-v4
 198  goodsn-wide1-v4
 581  goodsn-wide2-v4
 587  goodsn-wide3-v4
 565  goodsn-wide6-v4
 581  goodsn-wide66-v4
 567  goodsn-wide7-v4
 572  goodsn-wide8-v4
 610  gto-wide-cos01-v4
 570  gto-wide-cos02-v4
 589  gto-wide-cos03-v4
 575  gto-wide-cos04-v4
 573  gto-wide-cos05-v4
2034  gto-wide-egs1-v4
 594  gto-wide-egs2-v4
 596  gto-wide-uds10-v4
 592  gto-wide-uds11-v4
 594  gto-wide-uds12-v4
 609  gto-wide-uds13-v4
 583  gto-wide-uds14-v4
 126  iras16293-v4
 512  j0226-wang-v4
 161  j0252m0503-hennawi-02-v4
 159  j0252m0503-hennawi-07-v4
 399  j0910-wang-v4
 234  j1007p2115-hennawi-v4
  92  j1148-eilers-v4
  34  j1342-msa-v4
 774  jades-gdn-v4
1253  jades-gdn09-v4
1286  jades-gdn10-v4
1284  jades-gdn11-v4
 837  jades-gdn198-v4
3473  jades-gdn2-v4
 570  jades-gds-w03-v4
 600  jades-gds-w04-v4
 578  jades-gds-w05-v4
 579  jades-gds-w06-v4
 570  jades-gds-w07-v4
 588  jades-gds-w08-v4
 724  jades-gds-w09-v4
2979  jades-gds-wide-v4
1853  jades-gds-wide2-v4
 953  jades-gds-wide3-v4
 983  jades-gds02-v4
 970  jades-gds03-v4
1014  jades-gds04-v4
1148  jades-gds05-v4
 968  jades-gds06-v4
 978  jades-gds07-v4
1008  jades-gds08-v4
 924  jades-gds1-v4
 835  jades-gds10-v4
 100  lyc22-schaerer-01-v4
  80  lyc22-schaerer-03-v4
 104  lyc22-schaerer-12-v4
 583  macs0416-nakajima-v4
 284  macs0416-v4
 318  macs0417-v4
 404  macs1149-stiavelli-v4
 149  macs1149-stiavelli2-v4
 381  macs1149-v4
 252  macs1423-v4
  44  macsj0647-hr-v4
 137  macsj0647-single-v4
 142  macsj0647-v4
 178  mom-uds01-v4
 182  mom-uds02-v4
 755  nexus-obs3-v4
 958  nexus-obs5-v4
 426  ngc628-adamo-v4
  28  pearls-transients-v4
 325  rubies-egs51-v4
 439  rubies-egs52-v4
 450  rubies-egs53-v4
 406  rubies-egs61-v4
 497  rubies-egs62-v4
 479  rubies-egs63-v4
 516  rubies-uds1-v4
 478  rubies-uds2-v4
 433  rubies-uds21-v4
 437  rubies-uds22-v4
 405  rubies-uds23-v4
 496  rubies-uds3-v4
 443  rubies-uds31-v4
 512  rubies-uds32-v4
 491  rubies-uds33-v4
 496  rubies-uds41-v4
 471  rubies-uds42-v4
 465  rubies-uds43-v4
 225  rxj2129-ddt-v4
  86  smacs0723-ero-v4
 123  snh0pe-v4 
  78  spt0615-v4
  92  stark-a1703-v4
 147  stark-rxcj2248-v4
  62  suspense-kriek-v4
 192  ulas-j1120-gto-v4
 154  uncover-61-v4
 188  uncover-62-v4
 559  uncover-v4
  62  valentino-cosmos02-v4
  46  valentino-cosmos03-v4
  67  valentino-cosmos04-v4
 184  valentino-egs-v4
  47  valentino-obs08-v4
  99  valentino-obs10-v4
  68  valentino-obs12-v4
  58  valentino-obs15-v4
  55  valentino-obs19-v4
 106  weisz-leoa-v4
  77  weisz-tucana-v4
 225  westerlund2-imf-v4
 454  whl0137-v4





<grizli.utils.Unique at 0x307cc9130>

Source counts by grade

Show magnitude, color, redshift distribution as a function of the visual classification grade:

  • 3: Robust redshift from one or more emission absorption features
  • 2: Ambiguous continuum features, perhaps only one line or low confidence lines
  • 1: No clear features in the spetrum to constrain the redshift
  • 0: Spectrum suffers some data quality issue and should
  • -1: (Spectrum did not have grade from visual inspection)
fig, axes = plt.subplots(4,2,figsize=(8,10), sharex=False, sharey=True)

colors = {0: 'magenta', 1: '0.5', 2: 'coral', 3: 'olive'}

# sub = is_rubies
sub = tab['ra'] > 0

sub = sub & True

sub &= tab['z_phot'].filled(-1) > 0
sub &= tab['grating'] == 'PRISM'

un = utils.Unique(tab[sub]['grade'].filled(-1))

blue = -2.5*np.log10(tab['phot_f150w_tot_1'] / tab['phot_f444w_tot_1'])

for i, c in enumerate([3,2,1,0]):

    kws = dict(
        c = np.sqrt(tab[sub][un[c]]['exptime']), vmin=900**0.5, vmax=(5*3600)**0.5, cmap='magma_r',
        # c = 'magenta',
        alpha=0.5, 
        label=f'Grade = {c}',
    )
    
    ax = axes[i][1]
    ax.scatter(blue,
               23.9 - 2.5*np.log10(tab['phot_f444w_tot_1']),
               c='0.8',
               alpha=0.2, 
               label=f'Grade = {c}',
    )

    sc = ax.scatter(blue[sub][un[c]],
               23.9 - 2.5*np.log10(tab[sub]['phot_f444w_tot_1'])[un[c]],
               **kws,
    )
    ax.grid()

    if i < 3:
        ax.set_xticklabels([])

    ax.set_xlim(-2.2, 5.2)
        
    ax = axes[i][0]
    
    ax.scatter(np.log(1+tab['z_phot']),
               23.9 - 2.5*np.log10(tab['phot_f444w_tot_1']),
               c='0.8',
               alpha=0.2, 
               label=f'Grade = {c}' + '\n' + f'N = {un[c].sum()}',
    )

    ax.scatter(np.log(1+tab[sub]['z_phot'][un[c]]),
               23.9 - 2.5*np.log10(tab[sub]['phot_f444w_tot_1'])[un[c]],
               **kws,
    )
    
    ax.grid()
    ax.text(
        0.95, 0.05,
        # f'Grade = {c}',
        f'Grade = {c}' + '\n' + f'N = {un[c].sum()}',
        ha='right', va='bottom', fontsize=9, transform=ax.transAxes)

    if i < 3:
        ax.set_xticklabels([])
    
    xt = [0, 1, 2, 3, 4, 5, 6, 8, 10, 12, 16]
    ax.set_xlim(0, np.log(1+17))

ax.set_ylabel('mag F444W')

cax = fig.add_axes

cax = fig.add_axes((0.9, 0.1, 0.02, 0.15))
cb = plt.colorbar(sc, cax=cax, orientation='vertical')
ct = [0.5, 1, 2, 4]
cb.set_ticks(np.sqrt(np.array(ct)*3600))
cb.set_ticklabels(ct)
cb.set_label('EXPTIME (h)')

ax.set_ylim(19, 31)

ax.set_xticks(np.log(1+np.array(xt)))
ax.set_xticklabels(xt)
ax.set_xlabel(r'$z_\mathrm{phot}$')

ax = axes[3][1]

ax.set_xlabel('F150W - F444W')

# ax.legend()

fig.tight_layout(pad=1)


   N  value     
====  ==========
4799          -1
 114           0
1697           1
 815           2
13703           3

png

PRISM sample for comparision

is_prism = (tab['grating'] == 'PRISM')

sample = is_prism & (tab['grade'] == 3)
sample &= (tab['z_best'] < 7)
sample &= (tab['zrf'] > 0) & (tab['z_best'] > 0)
sample &= tab['rest_153_frac'] > 0.8
sample &= tab['rest_154_frac'] > 0.8
sample &= tab['rest_155_frac'] > 0.8
sample.sum()

15468

# Compare the redshift where the emission line fits were performed to the 
# "best" redshift calculated for discrete unique sources
_ = eazy.utils.zphot_zspec(tab['zrf'][is_prism], tab['z_best'][is_prism], zmax=8)

png

Interpolate Halpha EQW from nearby filters

import eazy.filters
RES = eazy.filters.FilterFile()

fb, fr = 415, 416
#fb, fr = 155, 416
wb = RES[fb].pivot
wr = RES[fr].pivot

flamb = (1*u.microJansky).to(u.erg/u.second/u.cm**2/u.Angstrom, equivalencies=u.spectral_density(wb*u.Angstrom))
flamr = (1*u.microJansky).to(u.erg/u.second/u.cm**2/u.Angstrom, equivalencies=u.spectral_density(wr*u.Angstrom))

whtb = (1 - np.abs(wb - 6564.)/(wr-wb))# *flamb
whtr = (1 - np.abs(wr - 6564.)/(wr-wb))# *flamr

interp_flux = tab[f'rest_{fb}_flux']*whtb*flamb + tab[f'rest_{fr}_flux']*whtr*flamr

eqw = ((tab['line_ha_nii']*1.e-20*u.erg/u.second/u.cm**2 / (interp_flux / (1+tab['zline'])**1))).value

plt.scatter(
    (np.maximum(tab['eqw_ha_nii'], -100) / (1+tab['zline'])**1)[sample],
    eqw[sample], alpha=0.02
)
plt.plot([0.1, 1e7], [0.1, 1e7], color='r', alpha=0.5)
plt.loglog()
plt.grid()
plt.xlim(0.02, 1.e5); plt.ylim(0.02, 1.e5)
plt.xlabel(r'H$\alpha$ EQW, template fit')
plt.ylabel(r'H$\alpha$ EQW, line flux / estimated continuum')

if 1:
    print('Use interpolated EQW')
    eqw_lim = np.maximum(tab['line_ha_nii'], tab['line_ha_nii_err']*2) * 1.e-20*u.erg/u.second/u.cm**2 / (interp_flux / (1+tab['zline']))
    is_eqw_lim = tab['line_ha_nii_err']*2 > tab['line_ha_nii']
    eqw[is_eqw_lim] = eqw_lim.value[is_eqw_lim]
    tab['ha_eqw_with_limits'] = eqw
    tab['ha_eqw_is_limit'] = is_eqw_lim
    

Use interpolated EQW

png

Stellar population properties

  • Rest-frame colors
  • Stellar masses
  • …
UV = -2.5*np.log10(tab['phot_restU'] / tab['phot_restV'])
VJ = -2.5*np.log10(tab['phot_restV'] / tab['phot_restJ'])

UVs = -2.5*np.log10(tab['rest_153_flux'] / tab['rest_155_flux'])
BVs = -2.5*np.log10(tab['rest_154_flux'] / tab['rest_155_flux'])
VJs = -2.5*np.log10(tab['rest_155_flux'] / tab['rest_161_flux'])

eBVs = 2.5/np.log(10) * np.sqrt(
    (tab['rest_154_full_err'] / tab['rest_154_flux'])**2
    + (tab['rest_155_full_err'] / tab['rest_155_flux'])**2
)

ugs = -2.5*np.log10(tab['rest_414_flux'] / tab['rest_415_flux'])
gis = -2.5*np.log10(tab['rest_415_flux'] / tab['rest_416_flux'])

ok_BVs = (tab['rest_154_frac'] > 0.8) & (tab['rest_155_frac'] > 0.8)
ok_gis = (tab['rest_415_frac'] > 0.8) & (tab['rest_416_frac'] > 0.8)

ok_BVs &= eBVs < 0.1

dL = WMAP9.luminosity_distance(tab['zrf']).to('cm')

rest_fV = (tab['rest_155_flux']*u.microJansky).to(
    u.erg/u.second/u.cm**2/u.Angstrom,
    equivalencies=u.spectral_density(5500.*(1+tab['zrf'])*u.Angstrom)
)

rest_fi = (tab['rest_416_flux']*u.microJansky).to(
    u.erg/u.second/u.cm**2/u.Angstrom,
    equivalencies=u.spectral_density(RES[416].pivot * (1+tab['zrf'])*u.Angstrom)
)

LV = (rest_fV * 5500. * u.Angstrom * (1 + tab['zrf']) * 4 * np.pi * dL**2).to(u.Lsun)
Li = (rest_fi * RES[416].pivot * u.Angstrom * (1 + tab['zrf']) * 4 * np.pi * dL**2).to(u.Lsun)


# Crude M/Lv ~ B-V from Taylor et al. 2009 for getting a quick stellar mass from the spectrum

log_MLv = -0.734 + 1.404 * (BVs + 0.084)
MassV = log_MLv + np.log10(LV.value)

tab['Mass'] = MassV
tab['Mass'].format = '.2f'
tab['ok_Mass'] = ok_BVs

plt.scatter(
    np.log10(tab['phot_mass'][sample & ok_BVs]),
    MassV[sample & ok_BVs],
    alpha=0.1,
    c=tab['phot_Av'][sample & ok_BVs]
)

plt.plot([5, 12], [5, 12], color='magenta')
plt.grid()
plt.xlim(6, 12)
plt.ylim(6, 12)
plt.xlabel('stellar mass,  eazy photometry')
plt.ylabel(r'$\log M = \log L_V + \log M/L_V$' + '\n' + r'$\log M/L_V \propto (B-V)$')

Text(0, 0.5, '$\\log M = \\log L_V + \\log M/L_V$\n$\\log M/L_V \\propto (B-V)$')

png

Compare rest-frame colors

The table includes rest-frame bandpass flux densities 1) estimated from the broad-band photometry (at the photo-z) and 2) integrated directly through the spectra at the measured redshift.

The colors derived from the grizli/DJA photometry are those of the best-fit photo-z template combination, not a noisy interpolation, so they can show banding effects resulting from the discrete combination of templates.

fig, axes = plt.subplots(1,2,figsize=(8,5), sharex=True, sharey=True)

axes[0].scatter(
    VJ[sample], UV[sample], alpha=0.1,
    c=tab['ha_eqw_with_limits'][sample], vmin=0, vmax=200, cmap='RdYlBu'
)
axes[0].set_xlabel(r'$(V-J)$' + ', eazy template')
axes[0].set_ylabel(r'$(U-V)$')

axes[1].scatter(
    VJs[sample], UVs[sample], alpha=0.1,
    c=tab['ha_eqw_with_limits'][sample], vmin=0, vmax=200, cmap='RdYlBu'
)

sc = axes[1].scatter(
    VJs[sample][:1], UVs[sample][:1], alpha=0.5,
    c=tab['ha_eqw_with_limits'][sample][:1], vmin=0, vmax=200, cmap='RdYlBu'
)

axes[1].set_xlabel(r'$(V-J)$' + ', spectrum')

for ax in axes:
    ax.set_xlim(-1.2, 4.2)
    ax.set_ylim(-0.8, 4.2)
    ax.grid()

cax = fig.add_axes((0.85, 0.2, 0.02, 0.25))
cb = plt.colorbar(sc, cax=cax, orientation='vertical')
cb.set_label(r'EQW H$\alpha$')

fig.tight_layout(pad=1)

png

plt.scatter(
    tab['z_best'][sample],
    # np.log10(tab['phot_mass'])[sample],
    tab['Mass'][sample],
    alpha=0.1,
    c=tab['ha_eqw_with_limits'][sample], vmin=0, vmax=200, cmap='RdYlBu'
)
plt.ylim(7, 12)
plt.grid()
plt.xlabel('redshift')
plt.ylabel('rough stellar mass')

Text(0, 0.5, 'rough stellar mass')

png

Make a table showing thumbnail and spectrum previews of a selected subsample

Here make a “massive galaxies” subsample with

  • grating=PRISM
  • z > 3
  • log M > 10.5
  • Spectrum covers rest-frame $B-V$

The preview table shows the first 32 of these, which actually tend to be quasars / LRDs where the stellar mass is likely incorrect….

massive = sample & (tab['z_best'] > 3.) & (MassV > 10.5) & (tab['grating'] == 'PRISM') & ok_BVs

if 0:
    tab['root','file','z_best','Mass','ha_eqw_with_limits','Thumb','Slit_Thumb','Spectrum_fnu', 'Spectrum_flam'][massive].write_sortable_html(
        '/tmp/massive.html',
        max_lines=1000,
        localhost=False,
    )
    
print(f"massive test sample: {massive.sum()}")

massive test sample: 235

from IPython.display import display, Markdown, Latex

so = np.argsort(tab['Mass'][massive])[::-1]
so = so[:32]

df = tab['root','file','z_best','Mass','ha_eqw_with_limits','Thumb','Slit_Thumb','Spectrum_fnu', 'Spectrum_flam'][massive][so].to_pandas()

display(Markdown(df.to_markdown()))
  root file z_best Mass ha_eqw_with_limits Thumb Slit_Thumb Spectrum_fnu Spectrum_flam
0 j0910-wang-v4 j0910-wang-v4_prism-clear_2028_12910.spec.fits 6.62142 12.0515 54.4902
1 rubies-uds23-v4 rubies-uds23-v4_prism-clear_4233_166691.spec.fits 4.06673 11.9254 11.4969
2 uncover-61-v4 uncover-61-v4_prism-clear_2561_13416.spec.fits 4.02262 11.7896 114.235
3 jades-gdn198-v4 jades-gdn198-v4_prism-clear_1181_68797.spec.fits 5.0398 11.7363 1029.12
4 rubies-egs52-v4 rubies-egs52-v4_prism-clear_4233_9809.spec.fits 5.68123 11.6563 197.61
5 jades-gdn-v4 jades-gdn-v4_prism-clear_1181_68797.spec.fits 5.0398 11.6303 1054.65
6 rubies-uds22-v4 rubies-uds22-v4_prism-clear_4233_114988.spec.fits 4.36474 11.5051 94.5801
7 rubies-uds1-v4 rubies-uds1-v4_prism-clear_4233_40579.spec.fits 3.10671 11.4453 1206.13
8 gto-wide-uds13-v4 gto-wide-uds13-v4_prism-clear_1215_1472.spec.fits 4.55596 11.4348 14.736
9 jades-gds-w05-v4 jades-gds-w05-v4_prism-clear_1212_4582.spec.fits 3.06306 11.3624 79.7105
10 jades-gds-wide3-v4 jades-gds-wide3-v4_prism-clear_1180_197911.spec.fits 3.06306 11.3613 80.9861
11 uncover-v4 uncover-v4_prism-clear_2561_45924.spec.fits 4.4673 11.3522 104.309
12 rubies-uds2-v4 rubies-uds2-v4_prism-clear_4233_40579.spec.fits 3.10671 11.3515 1369.53
13 capers-cos07-v4 capers-cos07-v4_prism-clear_6368_105080.spec.fits 5.58011 11.3182 138.384
14 glazebrook-v4 glazebrook-v4_prism-clear_2565_41232.spec.fits 3.12015 11.2815 17.5962
15 nexus-obs3-v4 nexus-obs3-v4_prism-clear_5105_23192.spec.fits 4.52105 11.2767 1058.6
16 capers-egs65-v4 capers-egs65-v4_prism-clear_6368_27615.spec.fits 5.68123 11.2409 241.543
17 uncover-v4 uncover-v4_prism-clear_2561_45092.spec.fits 3.46158 11.2391 98.5119
18 uncover-62-v4 uncover-62-v4_prism-clear_2561_57618.spec.fits 3.46158 11.2264 82.8891
19 uncover-62-v4 uncover-62-v4_prism-clear_2561_58453.spec.fits 4.4673 11.2252 969.994
20 gds-barrufet-s67-v4 gds-barrufet-s67-v4_prism-clear_2198_1260.spec.fits 4.4319 11.2251 72.7017
21 glazebrook-cos-obs3-v4 glazebrook-cos-obs3-v4_prism-clear_2565_20115.spec.fits 3.71293 11.2231 2.61723
22 glazebrook-v4 glazebrook-v4_prism-clear_2565_12629.spec.fits 3.19399 11.2039 5.4745
23 rubies-egs63-v4 rubies-egs63-v4_prism-clear_4233_49140.spec.fits 6.68847 11.1816 657.49
24 jades-gds-w08-v4 jades-gds-w08-v4_prism-clear_1212_792.spec.fits 3.67516 11.173 341.192
25 rubies-uds42-v4 rubies-uds42-v4_prism-clear_4233_807469.spec.fits 6.77538 11.1462 4951.05
26 rubies-egs53-v4 rubies-egs53-v4_prism-clear_4233_25712.spec.fits 3.89179 11.1396 120.361
27 uncover-v4 uncover-v4_prism-clear_2561_23955.spec.fits 3.47275 11.1197 131.122
28 goodsn-wide66-v4 goodsn-wide66-v4_prism-clear_1211_3184.spec.fits 3.4402 11.1181 59.2013
29 rubies-egs61-v4 rubies-egs61-v4_prism-clear_4233_55604.spec.fits 6.98435 11.1174 2762.98
30 uncover-61-v4 uncover-61-v4_prism-clear_2561_32864.spec.fits 3.05744 11.1156 148.446
31 jades-gds-wide3-v4 jades-gds-wide3-v4_prism-clear_1180_209777.spec.fits 3.70992 11.1048 244.425

Read a spectrum

The spectra can be accessed based on the root and file columns in the summary table.

ruby

import msaexp.spectrum

# Set spec_file here, will be used below in NN demo
spec_file = 'rubies-egs61-v4_prism-clear_4233_75646.spec.fits'

row = tab[tab['file'] == spec_file][0]
spec = msaexp.spectrum.SpectrumSampler(FITS_URL.format(**row))

row['Mass']

10.873998434439217

spec.spec.info

<GTable length=473>
    name     dtype  unit                description                   class     n_bad
----------- ------- ---- ----------------------------------------- ------------ -----
       wave float64                                                      Column     0
       flux float64                                                      Column     0
        err float64                                                      Column     0
        sky float64  uJy                                           MaskedColumn     5
  path_corr float64                                                MaskedColumn     5
       npix float64                                                      Column     0
   flux_sum float64                                                      Column     0
profile_sum float64                                                      Column     0
    var_sum float64                                                      Column     0
       corr float64                                                      Column     0
     escale float64                                                      Column     0
   full_err float64  uJy                                                 Column     0
      valid    bool                                                      Column     0
          R float64      Spectral resolution from tabulated curves       Column     0
    to_flam float64                                                      Column     0

plt.plot(spec['wave'], spec['flux'],
         label="{file}\nz={z_best:.3f}".format(**row))
plt.legend()

<matplotlib.legend.Legend at 0x1046e1940>

png

All msaexp PRISM spectra in a single table

All of the 1D extracted spectra have been collated into single FITS tables.

combined_spectra_file = f"dja_msaexp_emission_lines_{version}.prism_spectra.fits"

if os.path.exists(combined_spectra_file):
    prism_spectra = utils.read_catalog(combined_spectra_file)
else:
    # Combined prism spectra in a single big table (595 Mb)
    prism_spectra = utils.read_catalog(
        download_file(
            f"{URL_PREFIX}/{combined_spectra_file}",
            cache=CACHE_DOWNLOADS
        ),
        format='fits',
    )

prism_spectra.info()

<GTable length=473>
   name    dtype   shape      class     n_bad 
--------- ------- -------- ------------ ------
     wave float64                Column      0
     flux float64 (34949,)       Column      0
      err float64 (34949,)       Column      0
      sky float64 (34949,) MaskedColumn 697133
path_corr float64 (34949,) MaskedColumn 697133
     npix   int64 (34949,)       Column      0
 full_err float64 (34949,)       Column      0
    valid    bool (34949,)       Column      0

# The columns of the spectrum have N entries for N objects with a particular grating
# and are aligned with the summary table for that grating

is_prism = tab['grating'] == 'PRISM'
tab['prism_idx'] = 0
tab['prism_idx'][is_prism] = np.arange(is_prism.sum())

print(f"""
File: {combined_spectra_file}\n
{prism_spectra['flux'].shape[0]} wavelength bins\n
PRISM spectra in the merged catalog: {is_prism.sum()}
PRISM spectra in the combined table: {prism_spectra['flux'].shape}
""")

File: dja_msaexp_emission_lines_v4.4.prism_spectra.fits

473 wavelength bins

PRISM spectra in the merged catalog: 34949
PRISM spectra in the combined table: (473, 34949)

# Subset of "valid" spectra defined at most wavelengths
valid_count = prism_spectra['valid'].sum(axis=0)
valid_spec = valid_count > (valid_count.max() - 64)
valid_spec.sum()

31720

row_idx = np.where(tab['file'] == spec_file)[0][0]
row = tab[row_idx]

plt.plot(
    spec['wave'], spec['flux'],
    lw=2, label='Single spectrum'
)

plt.plot(
    prism_spectra['wave'], prism_spectra['flux'][:, row['prism_idx']],
    alpha=0.5, label='From combined table'
)

plt.legend()

<matplotlib.legend.Legend at 0x3baa4ea50>

png

“Stacked” spectrum

Stacking prism spectra isn’t trivial due to the variable dispersion and wavelength sampling. Here just plot a subset on top of each other.

norm_column = 'rest_416_flux'
print(f"Normalization column: '{norm_column}' = {tab[norm_column].description}")

flux_norm = prism_spectra['flux'] / tab[norm_column][is_prism]

# Subset
zi = row['z_best']
dz = 0.05

sample = (tab['z_best'] > zi - dz) & (tab['z_best'] < zi + dz)

sub_sample = sample[is_prism] & valid_spec
sub_idx = np.where(sub_sample)[0]

z_sample = tab['z_best'][is_prism][sample[is_prism] & valid_spec]

fig, axes = plt.subplots(2,1,figsize=(10,7), sharex=False, sharey=True)

for j, z in enumerate(z_sample):
    axes[0].plot(
        prism_spectra['wave'],
        flux_norm[:, sub_idx[j]],
        alpha=0.1
    )
    
    axes[1].plot(
        prism_spectra['wave'] / (1 + z),
        flux_norm[:, sub_idx[j]],
        alpha=0.1
    )

axes[0].set_ylim(-1, 10)

Normalization column: 'rest_416_flux' = Spectrum flux in synthetic_i





(-1.0, 10.0)

png

“Nearest neighbor” spectra

Simple “nearest neighbors” of the observed-frame normalized spectra extracted from a KDTree.

fix_flux_norm = flux_norm*1.
fix_flux_norm[~np.isfinite(flux_norm)] = 0

tr = cKDTree(fix_flux_norm[:, valid_spec].T)

N_nn = 32

row = tab[tab['file'] == spec_file][0]

# Features matrix is full normalized observed-frame spectra
Xfeatures = fix_flux_norm

tr_ds, tr_idx = tr.query(Xfeatures[:, row['prism_idx']], k=N_nn)

display_columns = [
    'root','file','zrf','Mass','ha_eqw_with_limits','Thumb','Slit_Thumb','Spectrum_fnu', 'Spectrum_flam'
]

df = tab[display_columns][is_prism][valid_spec][tr_idx][:16].to_pandas()

display(Markdown(df.to_markdown()))
  root file zrf Mass ha_eqw_with_limits Thumb Slit_Thumb Spectrum_fnu Spectrum_flam
0 rubies-egs61-v4 rubies-egs61-v4_prism-clear_4233_75646.spec.fits 4.90238 10.874 24.3336
1 gds-barrufet-s67-v4 gds-barrufet-s67-v4_prism-clear_2198_8777.spec.fits 4.65301 10.6769 8.26113
2 gds-barrufet-s67-v4 gds-barrufet-s67-v4_prism-clear_2198_8290.spec.fits 4.34034 10.5556 4.0581
3 mom-uds02-v4 mom-uds02-v4_prism-clear_5224_144670.spec.fits 3.97196 10.6169 14.667
4 jades-gdn09-v4 jades-gdn09-v4_prism-clear_1181_72127.spec.fits 4.13556 10.5771 75.7974
5 glazebrook-cos-obs1-v4 glazebrook-cos-obs1-v4_prism-clear_2565_10559.spec.fits 4.28971 10.5518 3.80496
6 jades-gds-wide-v4 jades-gds-wide-v4_prism-clear_1180_12619.spec.fits 3.60465 10.6324 27.8163
7 rubies-uds2-v4 rubies-uds2-v4_prism-clear_b28.spec.fits 4.39429 10.8802 5.61715
8 rubies-uds2-v4 rubies-uds2-v4_prism-clear_4233_b28.spec.fits 4.39461 10.8801 5.59215
9 glazebrook-cos-obs3-v4 glazebrook-cos-obs3-v4_prism-clear_2565_20115.spec.fits 3.72605 11.2231 2.61723
10 gto-wide-egs1-v4 gto-wide-egs1-v4_prism-clear_1213_4358.spec.fits 4.29302 10.6918 21.9825
11 jades-gdn09-v4 jades-gdn09-v4_prism-clear_1181_80660.spec.fits 4.40673 10.2177 25.751
12 glazebrook-v4 glazebrook-v4_prism-clear_2565_10459.spec.fits 3.97084 10.6829 4.65304
13 jades-gdn198-v4 jades-gdn198-v4_prism-clear_1181_76320.spec.fits 3.24558 10.306 5.3213
14 macs1149-v4 macs1149-v4_prism-clear_1208_5103925.spec.fits 3.69631 10.8235 6.45299
15 jades-gdn-v4 jades-gdn-v4_prism-clear_1181_76320.spec.fits 3.23434 10.3316 4.69578

Compute NN in rest-frame

The example above computed “raw” nearest neighbors from the full observed-frame spectra. To compute NN in the rest-frame, interpolate spectra to a fixed rest-frame wavelength grid.

from tqdm import tqdm

# rest-frame interpolated
wrest = utils.log_zgrid([0.08, 1.4], 1./1200/np.log(10))
wrest = np.unique([prism_spectra['wave'] / (1+z) for z in utils.log_zgrid([0.5, 7], 0.3)])
print(wrest.shape)

zero = wrest*0.

rest_flux_norm = []
for i, z in tqdm(enumerate(tab['zrf'][is_prism])):
    if z < 0:
        rest_flux_norm.append(zero)
    else:
        rest_flux_norm.append(np.interp(wrest, prism_spectra['wave'] / (1 + z), flux_norm[:,i], left=0, right=0))

rest_flux_norm = np.array(rest_flux_norm).T
rest_flux_norm[~np.isfinite(rest_flux_norm)] = 0

(2838,)


34949it [00:02, 14248.53it/s]

fig, ax = plt.subplots(1,1,figsize=(8,5))

ax.plot(
    prism_spectra['wave'] / (1 + tab['zrf'][row_idx]), flux_norm[:, row['prism_idx']],
    alpha=0.5, label='From combined table',
    color='0.5',
)

valid_rest = rest_flux_norm[:, row['prism_idx']] != 0
ax.scatter(
    wrest[valid_rest],
    rest_flux_norm[:, row['prism_idx']][valid_rest],
    marker='.',
    color='purple',
    label='resampled rest-frame',
    alpha=0.5
)

ax.legend()

fig.tight_layout(pad=1)

png

# Compute nearest-neighbors over limited rest-frame wavelength range
sli = np.where((wrest > 0.2) & (wrest < 0.8))[0]

Xfeatures = rest_flux_norm[sli,:]
tr = cKDTree(Xfeatures[:, valid_spec].T)


if False:
    # Trimmed spectrum and include log(1+z) as a feature
    sli = slice(32, -16)
    
    fix_flux_norm = np.vstack([flux_norm[sli,:]**1, np.log(1+tab['zrf'][is_prism]) * 1e-2]) 
    fix_flux_norm[~np.isfinite(fix_flux_norm)] = 0
    
    Xfeatures = fix_flux_norm
    tr = cKDTree(Xfeatures[:, valid_spec].T)


row = tab[tab['file'] == spec_file][0]

tr_ds, tr_idx = tr.query(Xfeatures[:, row['prism_idx']], k=N_nn)

df = tab[display_columns][is_prism][valid_spec][tr_idx][:16].to_pandas()

display(Markdown(df.to_markdown()))
  root file zrf Mass ha_eqw_with_limits Thumb Slit_Thumb Spectrum_fnu Spectrum_flam
0 rubies-egs61-v4 rubies-egs61-v4_prism-clear_4233_75646.spec.fits 4.90238 10.874 24.3336
1 ceers-v4 ceers-v4_prism-clear_1345_2759.spec.fits 3.43971 10.622 31.5176
2 capers-cos16-v4 capers-cos16-v4_prism-clear_6368_24202.spec.fits 3.09955 10.9213 20.8004
3 mom-uds02-v4 mom-uds02-v4_prism-clear_5224_144670.spec.fits 3.97196 10.6169 14.667
4 nexus-obs5-v4 nexus-obs5-v4_prism-clear_5105_27813.spec.fits 3.96582 11.0988 8.80705
5 jades-gds05-v4 jades-gds05-v4_prism-clear_1286_194373.spec.fits 2.67694 10.2399 42.391
6 ceers-ddt-v4 ceers-ddt-v4_prism-clear_2750_307.spec.fits 2.93255 10.9374 2.72842
7 glazebrook-egs-v4 glazebrook-egs-v4_prism-clear_2565_18996.spec.fits 3.23542 10.9906 23.3237
8 capers-egs49-v4 capers-egs49-v4_prism-clear_6368_7806.spec.fits 3.45255 10.285 33.2528
9 ceers-v4 ceers-v4_prism-clear_1345_2779.spec.fits 3.23845 10.8818 31.1324
10 snh0pe-v4 snh0pe-v4_prism-clear_4446_274.spec.fits 4.11111 10.8132 4.80224
11 egs-nelsonx-v4 egs-nelsonx-v4_prism-clear_4106_76085.spec.fits 3.22192 10.5733 34.906
12 rubies-egs63-v4 rubies-egs63-v4_prism-clear_4233_58841.spec.fits 3.45082 10.4517 27.5816
13 rubies-egs63-v4 rubies-egs63-v4_prism-clear_4233_61168.spec.fits 3.43539 10.8289 3.56196
14 glazebrook-egs-v4 glazebrook-egs-v4_prism-clear_2565_31322.spec.fits 3.4242 10.961 3.01521
15 jades-gds03-v4 jades-gds03-v4_prism-clear_1286_10026167.spec.fits 3.50452 10.3615 2.95476 
nn_sample = tab['ra'] < 0
nn_j = np.where(is_prism)[0][valid_spec][tr_idx]

nn_sample[nn_j] = True
print(nn_sample.sum())
sub_sample = nn_sample[is_prism] & valid_spec
sub_idx = np.where(sub_sample)[0]

z_sample = tab['zrf'][is_prism][nn_sample[is_prism] & valid_spec]
file_sample = tab['file'][is_prism][nn_sample[is_prism] & valid_spec]

fig, axes = plt.subplots(2,1,figsize=(10,7), sharex=False, sharey=True)

flam = -2
if 0:
    flam = 0

for j, z in enumerate(z_sample):
    kws = dict(
        alpha=0.3 if file_sample[j] == spec_file else 0.1,
        color='k' if file_sample[j] == spec_file else plt.cm.plasma(j/len(z_sample)),
        label=spec_file if file_sample[j] == spec_file else None,
        zorder=1000 if file_sample[j] == spec_file else 10,
    )
    
    axes[0].plot(
        # prism_spectra['wave'],
        # (flux_norm[:, sub_idx[j]] * (prism_spectra['wave'] / (1 + z) / 0.7)**flam),
        (Xfeatures[:, sub_idx[j]]), # * (prism_spectra['wave'] / (1 + z) / 0.7)**flam),
        **kws
    )
    
    axes[1].plot(
        prism_spectra['wave'] / (1 + z),
        (flux_norm[:, sub_idx[j]] * (prism_spectra['wave'] / (1 + z) / 0.7)**flam),
        **kws
    )

axes[0].set_xlabel(r'$i$')
axes[0].set_ylabel(r'$X_i$')

ymax = 2.2
axes[1].legend(loc='upper right')
axes[1].set_xlabel(r'$\lambda_\mathrm{rest}$')
axes[1].set_ylabel('normalized spectrum')

axes[0].set_ylim(-0.1*ymax, ymax)
for ax in axes:
    ax.grid()
    
fig.tight_layout(pad=1)

32

png

fig, ax = plt.subplots(1,1,figsize=(9,6))
tr_spec = (rest_flux_norm[:, valid_spec][:,tr_idx].T * (wrest/0.7)**flam).T
tr_spec[np.abs(tr_spec) < 1.e-6] = np.nan
tr_spec = tr_spec[:,:16]

xpl = wrest
xpl = np.arange(len(xpl))

_ = ax.plot(xpl, tr_spec[:,1:], alpha=0.1, color='0.7')

# _ = ax.plot(wrest, np.nanmean(tr_spec, axis=1), alpha=0.5, color='k')
_ = ax.plot(xpl, np.nanmedian(tr_spec[:,1:], axis=1), alpha=0.5, color='k', label='NN median')

src_spec = tr_spec[:,0] #np.nanmean((rest_flux_norm[:, valid_spec][sli,:][:,tr_idx[:1]].T * (wrest[sli]/0.7)**flam).T, axis=1)

_ = ax.plot(xpl, src_spec, alpha=0.6, color='tomato', label=spec_file)

ymax = 1.5*np.nanpercentile(src_spec, 90)

xt = np.append(np.arange(0.1, 1.01, 0.1), np.arange(1.2, 1.81, 0.2))
ax.set_xticks(np.interp(xt, wrest, xpl))
ax.set_xticklabels([f'{v:.1f}' for v in xt])

# ax.set_xlim(*np.interp([0.3, 1.9], wrest, xpl))

ax.legend(loc='upper right')

ax.set_ylim(-0.1*ymax, ymax)
ax.grid()
# ax.semilogx()
ax.set_ylabel(r'$f_\lambda$')
ax.set_xlabel(r'$\lambda_\mathrm{rest}$')
fig.tight_layout(pad=1)

png

Merged 1D grating spectra

Read 1D grating arrays

try:
    _ = grating_spectra
except NameError:
    # Initialize
    grating_spectra = {}

for (grating, filter) in [('G140M', 'F070LP'), ('G235M','F170LP'), ('G395M', 'F290LP')]: 
    key = (grating, filter)
    
    grating_spectra_file = f"dja_msaexp_emission_lines_{version}.{grating}-{filter}_spectra.fits".lower()
    
    if key not in grating_spectra:
        print(f"{key}: load {grating_spectra_file}")
        
        if os.path.exists(grating_spectra_file):
            grating_spectra[key] = utils.read_catalog(grating_spectra_file)
        else:
            # Combined grating spectra in a single big table
            grating_spectra[key] = utils.read_catalog(
                download_file(
                    f"{URL_PREFIX}/{grating_spectra_file}",
                    cache=CACHE_DOWNLOADS
                ),
                format='fits',
            )
        
    else:
        print(f"{key} spectra already loaded from {grating_spectra_file}")

    is_grating = (tab["grating"] == grating) & (tab["filter"] == filter)

    print(f"""
    {grating_spectra[key]["flux"].shape[0]} wavelength bins\n
    {grating} {filter} spectra in the combined table: {grating_spectra[key]["flux"].shape}
    {grating} {filter} entries in the summary table:  {is_grating.sum()}
""")


('G140M', 'F070LP'): load dja_msaexp_emission_lines_v4.4.g140m-f070lp_spectra.fits

    4667 wavelength bins

    G140M F070LP spectra in the combined table: (4667, 5851)
    G140M F070LP entries in the summary table:  5851

('G235M', 'F170LP'): load dja_msaexp_emission_lines_v4.4.g235m-f170lp_spectra.fits

    3685 wavelength bins

    G235M F170LP spectra in the combined table: (3685, 8000)
    G235M F170LP entries in the summary table:  8000

('G395M', 'F290LP'): load dja_msaexp_emission_lines_v4.4.g395m-f290lp_spectra.fits

    1661 wavelength bins

    G395M F290LP spectra in the combined table: (1661, 13606)
    G395M F290LP entries in the summary table:  13606

NN with grating spectra

Look for grating spectra of sources identified as nearest-neighbors above, matching on the obsid unique identifier.

nn_objid = tab['objid'][is_prism][valid_spec][tr_idx]
# gratings = np.unique(tab['grating'])
match_objid = np.isin(tab['objid'], nn_objid) & (tab['line_ha_err'] > 0)
_ = utils.Unique(tab['grating'][match_objid])
_ = utils.Unique(tab['filter'][match_objid])

   N  value     
====  ==========
   2  G140M     
   2  G235H     
   8  G235M     
   3  G395H     
  10  G395M     
   N  value     
====  ==========
   2  F070LP    
  10  F170LP    
  13  F290LP    

from scipy.stats import binned_statistic
import msaexp.utils

fig, axes = plt.subplots(1,3,figsize=(10,5), width_ratios=[0.6, 0.3, 1], sharey=True)

weighted_mean = {}

for key in grating_spectra:
    grating, filter = key

    if grating == 'G140M':
        continue
        
    full_spec = []

    in_grating = (tab['grating'] == grating) & (tab['filter'] == filter)

    # Sample with grating spectra that cover H-alpha
    nn_with_grating = np.isin(tab['objid'][in_grating], nn_objid)
    nn_with_grating &= (tab['line_ha_err'][in_grating] > 0) | (tab['line_oiii_5007_err'][in_grating] > 0)

    if nn_with_grating.sum() == 0:
        continue

    print(f'{grating}-{filter} N={nn_with_grating.sum()}')
    
    for j in np.where(nn_with_grating)[0]:

        wrest_j = grating_spectra[grating, filter]['wave'] / (1 + tab['z_best'][in_grating][j])
        
        flux_j = grating_spectra[grating, filter]['flux'][:,j] * 1
        err_j = grating_spectra[grating, filter]['err'][:,j] * 1 
        
        wsub = (wrest_j > 0.64) & (wrest_j < 0.68) & (err_j > 0)
        renorm_flux = np.median(flux_j[wsub])
        
        if 0:
            # Normalize to prism i band
            k = is_prism & (tab['objid'] == tab['objid'][in_grating][j])
            renorm_flux = np.nanmean(tab['rest_415_flux'][k])
        
        # print(renorm_flux, wsub.sum())
        # renorm_flux = 1.0
        
        flux_j /= renorm_flux
        err_j /= renorm_flux
        
        flux_j[err_j <= 0] = np.nan
        err_j[err_j <= 0] = np.nan

        # if np.nanmedian(err_j[wsub]) > 0.5:
        #     # Skip low S/N
        #     continue

        full_spec.append([wrest_j, flux_j, err_j])
        
        for ax in axes:
            ax.step(
                wrest_j,
                flux_j,
                alpha=0.02,
                zorder=-1,
            )

    # Bin by grating
    full_spec = np.array(full_spec)
    target_R = 1500
    wbin = 10**np.arange(*np.log10([0.3, 1.0]), 1./target_R/np.log(10))
    wbin_edge = msaexp.utils.array_to_bin_edges(wbin)

    wht = 1. / (full_spec[:,2,:]**2 + (0.1*full_spec[:,1,:])**2)
    
    num = binned_statistic(
        full_spec[:,0,:].flatten(),
        (full_spec[:,1,:] * wht).flatten(),
        bins=wbin_edge,
        statistic=np.nansum
    )
    
    denom = binned_statistic(
        full_spec[:,0,:].flatten(),
        wht.flatten(),
        bins=wbin_edge,
        statistic=np.nansum
    )

    wflux = num.statistic / denom.statistic
    werr = np.sqrt(1./denom.statistic)
    
    weighted_mean[key] = (wbin, wflux, werr)
    
    # optionally trim low S/N
    trim = wflux > 5 * werr
    
    for ax in axes:
        ax.step(wbin[trim], wflux[trim], color='k', alpha=0.3)

axes[0].set_xlim(0.31, 0.64)
axes[1].set_xlim(0.648, 0.665)
axes[2].set_xlim(0.62, 0.96)
ax.set_ylim(*(np.array([-0.03, 1.02])*2))

axes[0].set_ylabel(r'normalized $f_\nu$')
axes[1].set_xlabel(r'$\lambda_\mathrm{rest}$')

for ax in axes:
    ax.grid()

fig.tight_layout(pad=1)



G235M-F170LP N=9
G395M-F290LP N=10

png

Grating line fluxes

Do some simple scatter plots of line ratios

Simple OIII 4959/5007

has_line = (
    (tab['line_oiii_5007'] > 5 * tab['line_oiii_5007_err'])
    & (tab['line_oiii_4959_err'] > 0)
    & (tab['grade'] == 3)
    & np.isin(tab['grating'], ['G140M', 'G235M', 'G395M'])
)
print(f'OIII in grating spectra: {has_line.sum()}')

ung = utils.Unique(tab['grating'], verbose=False)

fig, ax = plt.subplots(1,1,figsize=(10,5))

for grating in ung.values:
    test = has_line & ung[grating]
    if test.sum() == 0:
        continue
        
    ax.scatter(
        tab['zrf'][test], (tab['line_oiii_4959'] / tab['line_oiii_5007'])[test],
        alpha=0.1,
        label=grating,
    )

ax.hlines(1./2.98, *ax.get_xlim(), color='k', ls=":", label="OIII 5007/4959 = 2.98")

ax.legend(loc='upper left')
ax.set_ylim(0, 1)
ax.grid()


OIII in grating spectra: 7040

png

Stellar mass vs grating [NII] / H$\alpha$

Rough tracer of the mass-metallicity relation.


dz = np.abs(tab['zrf'] - tab['z_phot']) / (1+tab['zrf'])

has_line = (
    (tab['line_ha'] > 3 * tab['line_ha_err'])
    & (tab['line_nii_6584_err'] > 0)
    & (tab['grade'] == 3)
    & np.isin(tab['grating'], ['G140M', 'G235M', 'G395M'])
    & (np.abs(dz) < 0.1)
)

print(f'Halpha in grating spectra: {has_line.sum()}')

fig, axes = plt.subplots(1,2,figsize=(8, 5), sharey=True)

scale_func = np.arcsinh

kws = dict(
    c=np.log(1+tab['zrf'][has_line]),
    cmap='rainbow',
    vmin=np.log(1+1.0), vmax=np.log(1+7),
    alpha=0.1
)

axes[0].scatter(
    np.log10(tab['phot_mass'])[has_line],
    scale_func(tab['line_nii_6584'] / tab['line_ha'])[has_line],
    **kws
)

axes[1].scatter(
    tab['zrf'][has_line],
    scale_func(tab['line_nii_6584'] / tab['line_ha'])[has_line],
    **kws
)

ax = axes[0]
yt = [-0.5, 0, 0.5] + list(range(6))[1:]
ax.set_yticks(scale_func(yt))
ax.set_yticklabels(yt)

ax.set_ylim(*scale_func([-0.6, 4]))

ax.set_xlim(5.9, 11.9)
ax.set_xlabel(r'$\log M/M_\odot$ (from photometry)')
ax.set_ylabel(r'[NII]$_{6584}$ / H$\alpha$')
axes[1].set_xlabel('redshift')

for ax in axes:
    ax.grid()

fig.tight_layout(pad=1)


Halpha in grating spectra: 6476

png

Thumbnail API

The DJA thumbnail API can create thumbnail figures and FITS cutouts of a requested set of filters at a particular coordinate.M

from IPython.display import Image
print(RGB_URL.format(**row))
Image(url=RGB_URL.format(**row), height=300, width=300)


https://grizli-cutout.herokuapp.com/thumb?size=1.5&scl=2.0&asinh=True&filters=f115w-clear%2Cf277w-clear%2Cf444w-clear&rgb_scl=1.5%2C0.74%2C1.3&pl=2&coord=214.91554591%2C52.94901831


print(SLIT_URL.format(**row))
Image(url=SLIT_URL.format(**row), height=300, width=300)

https://grizli-cutout.herokuapp.com/thumb?size=1.5&scl=4.0&invert=True&filters=f444w-clear&rgb_scl=1.5%2C0.74%2C1.3&pl=2&coord=214.91554591%2C52.94901831&nirspec=True&dpi_scale=6&nrs_lw=0.5&nrs_alpha=0.8&metafile=jw04233006001

]]>Gabriel BrammerSourceExtractor++ Morphological Catalogs2024-08-16T11:36:32+00:002024-08-16T11:36:32+00:00https://dawn-cph.github.io/dja/blog/2024/08/16/morphological-data imaging release catalog jwst

data-model-resid

Here we summarize the files available for morphological data. This extension to the DJA provides valuable morphological data for over 340k galaxies in the following fields :

  • ceers-full-grizli-v7.2
  • gds-grizli-v7.2
  • gdn-grizli-v7.3
  • primer-uds-north-grizli-v7.2
  • primer-uds-south-grizli-v7.2
  • primer-cosmos-east-grizli-v7.0
  • primer-cosmos-west-grizli-v7.0

The morphologies of these galaxies have been measured using both Sérsic and Bulge+Disk model. It made use of SourceXtractor++ to run the model fitting.

A merged catalog of all of the processed fields is available at full-good_morpho-phot.fits.gz (1.2 Gb). The first extension provides the merged DJA photometry and stellar population properties estimated from the photometric redshift fit. The second extension provides the morphology measurements of the Sersic model fit and the third extension provides parameters of the Bulge+Disk (B+D) morphology fit.

File extensions

For a given field, the following files are available:

  • {field}_morpho-phot.fits.gz: Merged catalogs of DJA photometry, EAZY SED fitting (from the DJA) and SE++ fittings in Sérsic and B+D models. They are stored in 3 HDUs inside the FITS file (DJA phot+EAZY, Sérsic model, B+D model) where the rows of each table correspond to one another. A flag column was added in the merged DJA catalog to flag sources non-detected by SE++ (0), potential artifcats (1) and low SNR or magnitude above the limiting mag in each field (3). Sources considered good for phyiscal study have a flag value of 2.

  • (model|resid)_{fit}_{field}*.fits.gz: Model or residual images generated by SE++ with the fit model for different bands. They have the exact same WCS as the images on the DJA.

  • {field}*star_psf.psf: PSFs generated for each field in each filter band using PSFEx. They made use of a custom star selection to give better results than the native auto-selection from PSFEx. This selection is based on the MU_MAX/MAG_AUTO plane, a method adapted from Leauthaud et al. 2007, Section 3.6.

You will also find the following file, non-relative to a specific field:

  • full-good_morpho-phot.fits.gz: Joint catalog of all the morpho-phot catalogs in the different fields, keeping only the good sources (flag value of 2). Useful for plots without having to open every catalog individually

File organization

All the files for each field are available in links in the table below.

Field Catalog Sérsic images B+D images PSFs
ceers-full-grizli-v7.2 Catalog Sérsic B+D PSF
gds-grizli-v7.2 Catalog Sérsic B+D PSF
gdn-grizli-v7.3 Catalog Sérsic B+D PSF
primer-uds-north-grizli-v7.2 Catalog Sérsic B+D PSF
primer-uds-south-grizli-v7.2 Catalog Sérsic B+D PSF
primer-cosmos-west-grizli-v7.0 Catalog Sérsic B+D PSF
primer-cosmos-east-grizli-v7.0 Catalog Sérsic B+D PSF

Code and catalog creation

All the code used to create these catalogs can be found in the DJA-SEpp GitHub repository. It contains:

  • The code of the dja_sepp package, made and used to generated these catalogs.

  • Scripts to:
    • run PSFEx on the images available on the DJA to generated empirical PSFs,
    • tile the images for SourceXtractor++,
    • run SourceXtractor++,
    • perform these steps autonomously on AWS EC2 instances.
  • Notebooks to:
    • run locally PSFEx and SourceXtractor++,
    • merge the tiled catalogs and images generated by SourceXtractor++,
    • merge the morphological catalogs generated by SourceXtractor++ with photometric catalogs from the DJA,
    • create plots to analyze the resulting catalogs,
    • look at specific sources.

The README file in the GitHub repository presents the workflow to run the steps used for this release. It can be used to perform this morphological study on different fields available on the DJA, or be adapated to different needs.

Completeness

The following table presents the galaxies whose morphology has been measured in the different fields used for this release.

Field DJA Sérsic Bulge+Disk Both
CEERS 67035 52604 (78.5%) 59046 (88.1%) 51329 (76.6%)
GOODS-S 57355 44931 (78.3%) 52754 (92.0%) 44016 (76.7%)
GOODS-N 65481 53291 (81.4%) 58852 (89.9%) 51465 (78.6%)
PRIMER-UDS (N) 68857 58947 (85.6%) 67134 (97.5%) 57945 (84.2%)
PRIMER-UDS (S) 65864 57397 (87.1%) 64537 (98.0%) 56476 (85.7%)
PRIMER-COSMOS (E) 50655 42359 (83.6%) 48496 (95.7%) 41597 (82.1%)
PRIMER-COSMOS (W) 51362 40493 (78.8%) 46964 (91.4%) 39704 (77.3%)
Total 426609 350022 (82.0%) 397783 (93.2%) 342892 (80.4%)

Technical details

For more information please check the Internship report written by Aurélien Genin. You can find details on the PSF estimation (3.2), the models used (3.3.1), the workings of SourceXtractor++ (3.3.2), the implementation on AWS (3.4) and some premilinary results (4) .

]]>Aurélien GeninDJA NIRSpec MSA Extractions v22024-03-01T12:31:30+00:002024-03-01T12:31:30+00:00https://dawn-cph.github.io/dja/blog/2024/03/01/nirspec-extractions-v2 spectroscopy nirspec release

(This page is auto-generated from the Jupyter notebook nirspec-extractions-v2.ipynb.)

We have reprocessed all of the public NIRSpec datasets from the earlier DJA release with the latest jwst pipeline and calibration files and updated msaexp v0.7 release.

Some of the major differences with respect to the previous reductions are as follows:

  • Improved initial Level1 processing with snowballs identified and masked in individual exposure groups using the snowblind module. The uncal.fits files are downloaded from MAST and processed automatically with grizli.aws.recalibrate.do_recalibrate.
  • The 2D extraction and rectification is handled in msaexp.slit_combine. The main algorithmic difference of the v2 extractions is that the background is removed by taking differences of the original 2D slitlet cutouts before drizzle resampling, i.e., differences between the cross-dispersion nods.
  • Then the final spectra are combined with a 2D weighted nearest-neighbor resampling to rectify them along rows of the 2D array, but where all wavelength bins are fully independent, i.e., no correlated noise from drizzling.
  • The 2D profile for the optimal extraction is also determined along the curved traces of the original spectral cutouts and rebinned/rectified in the same way as the data. The final 1D spectrum is the optimal extraction using this profile.
  • msaexp now has its own path loss correction that uses the width of the fitted profile along with the predicted intra-shutter position. The final spectra tend to lie much closer to the scale set by the photometry than the previous versions, though for many applications you’d probably still want to scale to whatever photometry yourself.
  • For programs that obtain spectra of a particular source with multiple dispersers (e.g., prism and gratings), the centering and source width of the optimal extraction profile is determined from the spectrum with the highest median S/N (usually the prism) and extractions in the other gratings are forced with those profile parameters.

Data release

The compilation of extracted spectra and redshift measurements is provided at nirspec_graded_v2.html.

The redshift quality grades have been copied from those of the previous release where the redshift measurements themselves agree to within tight tolerances, and extractions from any new programs since the first release were visually inspected and graded.

Note

The v2 spectra are affected by a bug that causes the uncertainties in the derived products to be too large by roughly a factor of f = N^{1/4}, where N is the number of combined exposures grouped by source_name / detector / grating / filter / MSA plan. For a simple set of three exposures with the 3-Shutter-Nod dither pattern, f = 3^{1/4} ~ 1.3, i.e., the tabulated uncertainties are roughly 1.3× too large. The effect of the bug is larger for deeper programs; for typical UNCOVER extractions f = 18^{1/4} ~ 2. For more information, see msaexp PR#54.

Observing programs

Program DJA root N Grating
2756 abell2744-ddt-v2 118 PRISM-CLEAR
1810 bluejay-north-v2 519 G140M-F100LP
  bluejay-south-v2   G235M-F170LP
      G395M-F290LP
1747 borg-0314m6712-v2 205 PRISM-CLEAR
  borg-0859p4114-v2    
  borg-1033p5051-v2    
  borg-1437p5044-v2    
  borg-2203p1851-v2    
2750 ceers-ddt-v2 251 PRISM-CLEAR
1345 ceers-v2 2236 G140M-F100LP
      G235M-F170LP
      G395M-F290LP
      PRISM-CLEAR
1871 gdn-chisholm-v2 44 G235H-F170LP
      G395H-F290LP
2198 gds-barrufet-s156-v2 137 PRISM-CLEAR
  gds-barrufet-s67-v2    
1210 gds-deep-v2 1219 G140M-F070LP
      G235M-F170LP
      G395H-F290LP
      G395M-F290LP
      PRISM-CLEAR
6541 gds-egami-ddt-v2 345 PRISM-CLEAR
3215 gds-udeep-v2 821 G140M-F070LP
      G395M-F290LP
      PRISM-CLEAR
2565 glazebrook-cos-obs2-v2 466 PRISM-CLEAR
  glazebrook-cos-obs3-v2    
  glazebrook-v2    
1211 goodsn-wide-v2 186 PRISM-CLEAR
1181 jades-gdn2-v2 4176 G140M-F070LP
  jades-gdn-v2   G235M-F170LP
      G395H-F290LP
      G395M-F290LP
      PRISM-CLEAR
1286 jades-gds1-v2 916 G140M-F070LP
      G235M-F170LP
      G395H-F290LP
      G395M-F290LP
      PRISM-CLEAR
1180 jades-gds-wide2-v2 4779 G140M-F070LP
  jades-gds-wide-v2   G235M-F170LP
      G395M-F290LP
      PRISM-CLEAR
4246 macsj0647-hr-v2 44 G395H-F290LP
1433 macsj0647-v2 140 PRISM-CLEAR
4557 pearls-transients-v2 210 PRISM-CLEAR
4233 rubies-uds1-v2 1480 G395M-F290LP
  rubies-uds2-v2   PRISM-CLEAR
  rubies-uds3-v2    
2767 rxj2129-ddt-v2 241 G140M-F070LP
      G140M-F100LP
      PRISM-CLEAR
2736 smacs0723-ero-v2 102 G235M-F170LP
      G395M-F290LP
4446 snh0pe-v2 116 G140M-F100LP
      G235M-F170LP
      PRISM-CLEAR
2110 suspense-kriek-v2 45 G140M-F100LP
2561 uncover-v2 766 PRISM-CLEAR 
import os
if os.path.exists('gbrammer' in os.environ['HOME']) & False:
    
    # Extract summary from database
    from grizli.aws import db
    import pyperclip

    # db queries require DB credentials
    nre = db.SQL("""select SUBSTR(MIN(dataset), 4, 4) as pid,
                           array_agg(DISTINCT(root)) as root,
                           count(*) as N,
                           array_agg(DISTINCT(grating || '-' || filter)) as Grating
    FROM nirspec_extractions WHERE ROOT LIKE '%%v2' AND dataset not like 'jw01208%%'
    GROUP BY SUBSTR(dataset, 4, 4)
    ORDER BY MAX(root)""")
    
    nre['root'] = ['\n'.join(r) for r in nre['root']]
    nre['grating'] = ['\n'.join(r) for r in nre['grating']]
    nre.rename_column('root','DJA root')
    url = '[{pid}](https://www.stsci.edu/cgi-bin/get-proposal-info?id={pid}&observatory=JWST)'
    prog = [url.format(**row) for row in nre]
    nre['Program'] = prog
    
    pyperclip.copy('## Observing programs\n\n' +
                   nre['Program','DJA root','n','grating'].to_pandas(index=False).to_markdown(index=False))

%matplotlib inline

import os
import yaml

import numpy as np
import matplotlib.pyplot as plt

import warnings
warnings.filterwarnings('ignore')

import astropy.io.fits as pyfits

import grizli
import grizli.catalog
from grizli import utils

import eazy
import msaexp
import msaexp.spectrum

print(f'grizli version: {grizli.__version__}')
print(f'eazy-py version: {eazy.__version__}')
print(f'msaexp version: {msaexp.__version__}')

grizli version: 1.11.3.dev2+gd57387c.d20240222
eazy-py version: 0.6.8.dev1+g3fb0ad2.d20240129
msaexp version: 0.7.3.dev0+g3e012ec.d20240222

Compare v1 and v2

  • The v2 extractions are generally cleaner with fewer non-Gaussian outliers
  • The normalization of the v2 extractions is generally both “brighter” and “redder” as a result of the internal wavelength-dependent path loss correction
z = 3.7616 
_prefix = 'https://s3.amazonaws.com/msaexp-nirspec/extractions/'
v2_file = _prefix + 'ceers-ddt-v2/ceers-ddt-v2_prism-clear_2750_1598.spec.fits'

v1_file = v2_file.replace('ceers-v2','ceers-lr-v1').replace('-v2','-v1')
print(v1_file + '\n' + v2_file)

sp1 = msaexp.spectrum.SpectrumSampler(v1_file)
sp2 = msaexp.spectrum.SpectrumSampler(v2_file)

f1 = sp1.drizzled_hdu_figure(z=z, unit='fnu')
f2 = sp2.drizzled_hdu_figure(z=z, unit='fnu')

https://s3.amazonaws.com/msaexp-nirspec/extractions/ceers-ddt-v1/ceers-ddt-v1_prism-clear_2750_1598.spec.fits
https://s3.amazonaws.com/msaexp-nirspec/extractions/ceers-ddt-v2/ceers-ddt-v2_prism-clear_2750_1598.spec.fits


import msaexp.slit_combine
from msaexp.slit_combine import slit_prf_fraction

fig, axes = plt.subplots(2,1,figsize=(8,4), sharex=True)

ax = axes[0]
ax.plot(sp1['wave'], sp1['flux'], label='v1')
ax.plot(sp2['wave'], sp2['flux'], label='v2')
leg = ax.legend(loc='upper left')
leg.set_title(sp2.meta['SRCNAME'])
ax.grid()

# Ratios
axes[1].plot(sp2['wave'], sp2['flux'] / sp1['flux'], label=r'v2 / v1', color='pink')

path_corr = slit_prf_fraction(sp2.spec['wave'].astype(float),
                              sigma=sp2.spec.meta['SIGMA'],
                              x_pos=sp2.spec.meta['SRCXPOS'],
                              slit_width=0.2,
                              pixel_scale=msaexp.slit_combine.PIX_SCALE,
                              verbose=False)

axes[1].plot(sp2.spec['wave'], 1./path_corr, color='plum', label='v2 path loss correction')
axes[1].grid()
axes[1].legend(loc='upper left')

fig.tight_layout(pad=1)

Compare gratings

All v2 extractions with different gratings for a particular source + program now have the same root, rather than being split in some cases. For example, all of the CEERS (ERS-1345) spectra now have root = ceers-v2 where they were split between the prisms (ceers-lr-v1) and gratings (ceers-mr-v1) before.

Note: There remains an msaexp bug that causes the tick intervals on the automatic grating figures plotted below to look strange. The minor ticks are evenly spaced in 0.05µm intervals, but the labels are rounded to a single decimal place (e.g., 3.75 becomes 3.8).

z = 2.0611
_prefix = 'https://s3.amazonaws.com/msaexp-nirspec/extractions/'
prism_file = _prefix + 'ceers-v2/ceers-v2_prism-clear_1345_3506.spec.fits'

sp = {}
for grating in ['prism-clear','g140m-f100lp','g235m-f170lp','g395m-f290lp']:
    file = prism_file.replace('prism-clear',grating)
    print(file)
    sp[grating] = msaexp.spectrum.SpectrumSampler(file)

    fig = sp[grating].drizzled_hdu_figure(z=z, unit='fnu')

https://s3.amazonaws.com/msaexp-nirspec/extractions/ceers-v2/ceers-v2_prism-clear_1345_3506.spec.fits
https://s3.amazonaws.com/msaexp-nirspec/extractions/ceers-v2/ceers-v2_g140m-f100lp_1345_3506.spec.fits
https://s3.amazonaws.com/msaexp-nirspec/extractions/ceers-v2/ceers-v2_g235m-f170lp_1345_3506.spec.fits
https://s3.amazonaws.com/msaexp-nirspec/extractions/ceers-v2/ceers-v2_g395m-f290lp_1345_3506.spec.fits


fig, axes = plt.subplots(4,1,figsize=(8,8))

colors = ['0.2','steelblue','orange','tomato']

for i, grating in enumerate(sp):
    sp[grating]['flux'][~sp[grating].valid] = np.nan
    for ax in axes:
        ax.plot(sp[grating]['wave'], sp[grating]['flux'], alpha=0.5, label=grating, color=colors[i])

leg = axes[3].legend(ncol=4, fontsize=8)
axes[0].set_title(sp[grating].meta['SRCNAME'])

axes[0].set_xlim(0.9, 1.9); axes[0].set_ylim(-1.2, 12)
axes[1].set_xlim(1.85, 3.2); axes[1].set_ylim(-0.5, 5)
axes[2].set_xlim(2.95, 5.3); axes[2].set_ylim(-0.8, 8)
axes[3].set_ylim(-3,30)
axes[3].set_xlabel(r'$\lambda_\mathrm{obs}~[\mu\mathrm{m}]$')

for ax in axes:
    ax.grid()
    ax.set_ylabel(r'$f_\nu~[\mu\mathrm{Jy}]$')

fig.tight_layout(pad=1)

]]>Gabriel BrammerJWST Cycle 1 and 2 timeline2023-08-04T10:28:44+00:002023-08-04T10:28:44+00:00https://dawn-cph.github.io/dja/blog/2023/08/04/visit-timeline news features

A scrape of the Cycle 1 and 2 visit information pages is now available here.

]]>Gabriel BrammerDJA NIRSpec Spectroscopic Data Products2023-07-18T19:58:23+00:002023-07-18T19:58:23+00:00https://dawn-cph.github.io/dja/blog/2023/07/18/nirspec-data-products spectroscopy demo nirspec

(This page is auto-generated from the Jupyter notebook nirspec-data-products.ipynb.)

Here we summarize the files available for the DJA reductions of the public NIRSpec MSA datasets.

The full interactive table can be found here and with an associated CSV catalog nirspec_graded_v0.ecsv.

%matplotlib inline
import os
import warnings
warnings.filterwarnings('ignore')

import numpy as np
import matplotlib.pyplot as plt
import scipy.ndimage as nd

import astropy.io.fits as pyfits
import astropy.units as u

import sep

import grizli
from grizli import utils
print(f'grizli version: {grizli.__version__}')

import msaexp
import msaexp.spectrum
print(f'msaexp version: {msaexp.__version__}')

BASE_URL = 'https://s3.amazonaws.com/msaexp-nirspec/extractions/'
PATH_TO_FILE = BASE_URL + '{root}/{file}'

grizli version: 1.10.dev3+g341a999
msaexp version: 0.6.12.dev5+ge335c82.d20230530

Full summary catalog

grade based on visual inspection of individual spectra. That is, a particular object can have multiple entries in the table and only grade=3 for the grating+filter combination that showed robust features.

  • 3 = Robust
  • 2 = Perhaps line or continuum features, but ambiguous redshift
  • 1 = No features
  • 0 = DQ problem
nrs = utils.read_catalog(BASE_URL + 'nirspec_graded_v0.ecsv', format='ascii.ecsv')

print('# Grade')
un = utils.Unique(nrs['grade'])

# Grade
   N  value     
====  ==========
 381           0
 624           2
1810           3
2309           1

# By grating
grat = utils.Unique(nrs['grating'])

   N  value     
====  ==========
 176  G235H     
 179  G140H     
 331  G395H     
 574  G395M     
 598  G140M     
 607  G235M     
2659  PRISM     

# By project
root = utils.Unique(nrs['root'])

   N  value     
====  ==========
  69  macsj0647-single-v1
 101  smacs0723-ero-v1
 111  abell2744-ddt-v1
 117  snH0pe-v1 
 132  macsj0647-v1
 143  rxj2129-ddt-v1
 154  gds-deep-hr-v1
 182  goodsn-wide-v1
 266  ceers-ddt-v1
 294  gds-deep-lr-v1
 405  whl0137-v1
 532  abell2744-glass-v1
 654  gds-deep-mr-v1
 906  ceers-mr-v1
1058  ceers-lr-v1

# Robust redshifts
robust_prism = (nrs['grating'] == 'PRISM') & (nrs['grade'] == 3)

roots = utils.Unique(nrs['root'][robust_prism], verbose=False)

nx = 2
ny = int(np.ceil((len(roots) + 1) / 2))

sx = 2.5

fig, axes = plt.subplots(ny,nx,sharex=True, sharey=False, figsize=(nx*sx*2, ny*sx))

lnz = np.log(1+nrs['z'])

bins = utils.log_zgrid([0.1, 13], 0.1)

xtv = [0,1,2,3,4,5,6,7,8,9,10,11,12,13]
xtl = [0,1,2,3,4,5,6,7,8,9,10,'',12,'']

_ = axes[0][0].hist(lnz[robust_prism], bins=np.log(1+bins))
axes[0][0].set_ylabel('All PRISM')
axes[0][0].grid()

count = 0

for root in roots.values:
    count += 1
    i = count // ny
    j = count - ny*i
    
    _ = axes[j][i].hist(lnz[robust_prism][roots[root]], bins=np.log(1+bins))
    axes[j][i].set_ylabel(root)
    axes[j][i].grid()
    
axes[j][i].set_xticks(np.log(1+np.array(xtv)))
axes[j][i].set_xticklabels(xtl)

for last in range(count*1, nx*ny-1):
    count += 1
    i = count // ny
    j = count - ny*i
    
    _ = axes[j][i].axis('off')

_ = fig.tight_layout(pad=1)

png

Look at a source from JADES-DEEP

# id = '58975' # remarkable z=9
# ymax = [1, 0.2, 0.7, 0.7]
# xlimits = [(0.5, 5.4), (0.5, 2.2), (3.9, 4.6), (5.0, 5.3)]

# id = '35180' # bright z=1.8
# ymax = [2]*4

id = '15157' # z=4.1
ymax = [0.8, 0.4, 0.8, 0.8]
xlimits = [(0.5, 5.4), (0.5, 2.2), (2.2, 2.8), (3.18, 3.6)]

is_gds = np.array([f.startswith('gds-deep') for f in nrs['file']])

src = np.array([id in f for f in nrs['file']]) & is_gds

nrs['root','file','grating','filter','z','grade'][src]

urls = [PATH_TO_FILE.format(**row) for row in nrs[src]]

z = np.mean(nrs['z'][src & (nrs['grade'] == 3)])

nrs[src]['root','file','grating','filter','z','grade']
GTable length=5
rootfilegratingfilterzgrade
str19str53str5str6float64int64
gds-deep-hr-v1gds-deep-hr-v1_g395h-f290lp_1210_15157.spec.fitsG395HF290LP4.14863
gds-deep-lr-v1gds-deep-lr-v1_prism-clear_1210_15157.spec.fitsPRISMCLEAR4.14993
gds-deep-mr-v1gds-deep-mr-v1_g140m-f070lp_1210_15157.spec.fitsG140MF070LP4.21561
gds-deep-mr-v1gds-deep-mr-v1_g235m-f170lp_1210_15157.spec.fitsG235MF170LP4.14823
gds-deep-mr-v1gds-deep-mr-v1_g395m-f290lp_1210_15157.spec.fitsG395MF290LP4.14823

Slitlet viewer

URLs like:

https://grizli-cutout.herokuapp.com/thumb?size=1&scl=7.0&invert=True&filters=f444w-clear&pl=2&coord=53.113326%20-27.802992&nirspec=True&dpi_scale=6&nrs_source=magenta&nrs_other=pink&nrs_lw=0.5&nrs_alpha=0.8

2D spectra

for u in urls:
    if 'prism' in u:
        break

print(f'Load 2D file: {u}')

img = pyfits.open(u)
spec = utils.GTable(img['SPEC1D'].data)
img.info()

Load 2D file: https://s3.amazonaws.com/msaexp-nirspec/extractions/gds-deep-lr-v1/gds-deep-lr-v1_prism-clear_1210_15157.spec.fits
Filename: /Users/gbrammer/.astropy/cache/download/url/8f02d3866629baee18593ec239973c9c/contents
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU       4   ()      
  1  SPEC1D        1 BinTableHDU   1504   435R x 3C   ['D', 'D', 'D']   
  2  SCI           1 ImageHDU      1494   (435, 41)   float32   
  3  WHT           1 ImageHDU      1494   (435, 41)   float32   
  4  PROFILE       1 ImageHDU      1494   (435, 41)   float64   
  5  PROF1D        1 BinTableHDU     25   41R x 3C   [D, E, D]   

# Metadata and exposure data
for i, k in enumerate(img['SCI'].header):
    print(f"{k} : {img['SCI'].header[k]}")
    if k == 'FILE2':
        break
        
print('...')

XTENSION : IMAGE
BITPIX : -32
NAXIS : 2
NAXIS1 : 435
NAXIS2 : 41
PCOUNT : 0
GCOUNT : 1
CRPIX1 : 218
CRPIX2 : 21
CRPIX3 : 1
CRVAL1 : 34478.17210692866
CRVAL2 : 53.11342711704571
CRVAL3 : -27.80296418104931
CD1_1 : 112.0143042734245
CD2_2 : -2.5018683716805e-05
CD3_2 : 4.3106889543069e-06
CD2_3 : 1.0
CD3_3 : 1.0
CTYPE1 : WAVELEN
CTYPE2 : RA---TAN
CTYPE3 : DEC--TAN
CUNIT1 : Angstrom
CUNIT2 : deg
CUNIT3 : deg
RADESYS : ICRS
WCSNAME : SLITWCS
SLIT_PA : 99.77600798539926
PSCALE : 0.09145217434882043
SLIT_Y0 : 0.1046029202771379
SLIT_DY : -0.2
LMIN : 5.336618171269953
DLAM : 0.01120143042734245
BKGOFF : 6
OTHRESH : 30
WSAMPLE : 1.05
LOGWAVE : False
BUNIT : mJy
GRATING : PRISM
FILTER : CLEAR
NFILES : 36
EFFEXPTM : 99788.00399999997
SRCNAME : 1210_15157
SRCID : 15157
SRCRA : 53.113326
SRCDEC : -27.80299179999997
FILE1 : jw01210001001_11101_00001_nrs1_phot.247.1210_15157.fits
CALVER1 : 1.9.4
CRDS1 : jwst_1084.pmap
GRAT1 : PRISM
FILTER1 : CLEAR
MSAMET1 : jw01210001001_02_msa.fits
MSAID1 : 117
MSACNF1 : 2
SLITID1 : 247
SRCNAM1 : 1210_15157
SRCID1 : 15157
SRCRA1 : 53.113326
SRCDEC1 : -27.80299179999997
PIXSR1 : 4.83547881464665e-13
TOUJY1 : 0.483547881464665
DETECT1 : NRS1
XSTART1 : 1323
XSIZE1 : 425
YSTART1 : 233
YSIZE1 : 20
RA_REF1 : 53.1410916104911
DE_REF1 : -27.79159354885499
RL_REF1 : -38.97457220320062
V2_REF1 : 378.563202
V3_REF1 : -428.402832
V3YANG1 : 138.5745697
RA_V11 : 52.96411558412523
DEC_V11 : -27.76510758004405
PA_V31 : 321.1077987432105
EXPSTR1 : 59873.16264294121
EXPEND1 : 59873.16264294121
EFFEXP1 : 2771.889
DITHN1 : 1
DITHX1 : 7.646252570500947
DITHY1 : 29.2702219123212
DITHT1 : 3-SHUTTER-SLITLET
NFRAM1 : 5
NGRP1 : 19
NINTS1 : 2
RDPAT1 : NRSIRS2
FILE2 : jw01210001001_11101_00002_nrs1_phot.247.1210_15157.fits
...

fig, axes = plt.subplots(3,1, figsize=(8, 5), sharex=True, sharey=True)

msk = img['WHT'].data > 0

for i, k in enumerate(['SCI','WHT','PROFILE']):
    _data = img[k].data
    vm = np.percentile(_data[msk], [2,99])
    ax = axes[i]
    ax.imshow(_data, vmin=vm[0], vmax=vm[1], cmap='plasma_r', aspect='auto')
    ax.set_ylabel(k)
    ax.set_yticklabels([])
    
xtv = [1,2,3,4,5]
xti = np.interp(xtv, spec['wave'], np.arange(len(spec)))

a2 = axes[0].twiny()
a2.set_xlabel('pix')
a2.set_xlim(0, _data.shape[1])

ax.set_xticks(xti)
ax.set_xticklabels(xtv)

ax.set_xlabel(r'$\lambda_\mathrm{obs}\,[\mu\mathrm{m}]$')

_ = fig.tight_layout(pad=1)

png

Optimal extraction profile

The 1D spectrum is created with an “optimal” (Horne 1986) extraction where the profile was fit to the observed spectrum both in terms of the center and wavelength-dependent width.

# Collapsed 1D profile
p1 = utils.GTable(img['PROF1D'].data)

fig, ax = plt.subplots(1,1,figsize=(5,3))
ax.plot(p1['pix'], p1['profile'], label='data', color='k')
ax.plot(p1['pix'], p1['pfit'], label='fit', color='r', lw=4, alpha=0.5)

ax.set_xlabel(r'$\Delta y$, pix')
ax.set_ylabel('Collapsed profile')
ax.legend()

ax.grid()

png

wlims = [(1.8, 2.1), (5, 5.2)]
fig, ax = plt.subplots(1, 1, figsize=(8, 4), sharex=True)

w2d = np.ones(img['SCI'].data.shape)*spec['wave']

for i, wlim in enumerate(wlims):
    msk = (w2d > wlim[0]) & (w2d < wlim[1]) & (img['WHT'].data > 0)
    ydata = (img['SCI'].data*msk).sum(axis=1)
    pdata = (img['PROFILE'].data*msk).sum(axis=1)
    
    anorm = (ydata*pdata).sum()/(pdata**2).sum()
    pdata *= anorm
    norm = pdata.sum()
    
    pl = ax.plot(ydata/norm, label=r'$xx < \lambda < yy$'.replace('xx', f'{wlim[0]:.1f}').replace('yy', f'{wlim[1]:.1f}'))
    ax.plot(pdata/norm, label='profile', color=pl[0].get_color(), linestyle='--')
    
ax.set_ylim(-0.18, 0.5)
ax.legend()
ax.set_xlim(10,30)
ax.grid()

png

1D spectra

Some tools in msaexp.spectrum.SpectrumSampler for interacting with 1D spectra.

# Open the files into 1D objects, directly from the web
sobj = {}
for u in urls:
    print(f'Read {u}')
    key = os.path.basename(u)
    sobj[key] = msaexp.spectrum.SpectrumSampler(u)

Read https://s3.amazonaws.com/msaexp-nirspec/extractions/gds-deep-hr-v1/gds-deep-hr-v1_g395h-f290lp_1210_15157.spec.fits
Read https://s3.amazonaws.com/msaexp-nirspec/extractions/gds-deep-lr-v1/gds-deep-lr-v1_prism-clear_1210_15157.spec.fits
Read https://s3.amazonaws.com/msaexp-nirspec/extractions/gds-deep-mr-v1/gds-deep-mr-v1_g140m-f070lp_1210_15157.spec.fits
Read https://s3.amazonaws.com/msaexp-nirspec/extractions/gds-deep-mr-v1/gds-deep-mr-v1_g235m-f170lp_1210_15157.spec.fits
Read https://s3.amazonaws.com/msaexp-nirspec/extractions/gds-deep-mr-v1/gds-deep-mr-v1_g395m-f290lp_1210_15157.spec.fits

# A single (prism) spectrum

for i, k in enumerate(sobj):
    if 'prism' in k:
        break

sp = sobj[k]
sp.spec.info()

<GTable length=435>
  name    dtype  unit
-------- ------- ----
    wave float64     
    flux float64     
     err float64     
    corr float64     
  escale float64     
full_err float64  uJy
   valid    bool     
       R float64     
 to_flam float64     

# Metadata about the 1D extraction
for i, k in enumerate(sp.spec.meta):
    print(f"{k} : {sp.spec.meta[k]}")
    if k == 'CRDS2':
        break
        
print('...')

VERSION : 0.6.12.dev18+gc61a335
TOMUJY : 1.0
PROFCEN : -0.9085424632954114
PROFSIG : 1.263439656446174
PROFSTRT : 0
PROFSTOP : 435
YTRACE : 20.0
EXTNAME : SPEC1D
CRPIX1 : 218
CRPIX2 : 21
CRPIX3 : 1
CRVAL1 : 34478.17210692866
CRVAL2 : 53.11342711704571
CRVAL3 : -27.80296418104931
CD1_1 : 112.0143042734245
CD2_2 : -2.5018683716805e-05
CD3_2 : 4.3106889543069e-06
CD2_3 : 1.0
CD3_3 : 1.0
CTYPE1 : WAVELEN
CTYPE2 : RA---TAN
CTYPE3 : DEC--TAN
CUNIT1 : Angstrom
CUNIT2 : deg
CUNIT3 : deg
RADESYS : ICRS
WCSNAME : SLITWCS
SLIT_PA : 99.77600798539926
PSCALE : 0.09145217434882043
SLIT_Y0 : 0.1046029202771379
SLIT_DY : -0.2
LMIN : 5.336618171269953
DLAM : 0.01120143042734245
BKGOFF : 6
OTHRESH : 30
WSAMPLE : 1.05
LOGWAVE : False
BUNIT : mJy
GRATING : PRISM
FILTER : CLEAR
NFILES : 36
EFFEXPTM : 99788.00399999997
SRCNAME : 1210_15157
SRCID : 15157
SRCRA : 53.113326
SRCDEC : -27.80299179999997
FILE1 : jw01210001001_11101_00001_nrs1_phot.247.1210_15157.fits
CALVER1 : 1.9.4
CRDS1 : jwst_1084.pmap
GRAT1 : PRISM
FILTER1 : CLEAR
MSAMET1 : jw01210001001_02_msa.fits
MSAID1 : 117
MSACNF1 : 2
SLITID1 : 247
SRCNAM1 : 1210_15157
SRCID1 : 15157
SRCRA1 : 53.113326
SRCDEC1 : -27.80299179999997
PIXSR1 : 4.83547881464665e-13
TOUJY1 : 0.483547881464665
DETECT1 : NRS1
XSTART1 : 1323
XSIZE1 : 425
YSTART1 : 233
YSIZE1 : 20
RA_REF1 : 53.1410916104911
DE_REF1 : -27.79159354885499
RL_REF1 : -38.97457220320062
V2_REF1 : 378.563202
V3_REF1 : -428.402832
V3YANG1 : 138.5745697
RA_V11 : 52.96411558412523
DEC_V11 : -27.76510758004405
PA_V31 : 321.1077987432105
EXPSTR1 : 59873.16264294121
EXPEND1 : 59873.16264294121
EFFEXP1 : 2771.889
DITHN1 : 1
DITHX1 : 7.646252570500947
DITHY1 : 29.2702219123212
DITHT1 : 3-SHUTTER-SLITLET
NFRAM1 : 5
NGRP1 : 19
NINTS1 : 2
RDPAT1 : NRSIRS2
FILE2 : jw01210001001_11101_00002_nrs1_phot.247.1210_15157.fits
CALVER2 : 1.9.4
CRDS2 : jwst_1084.pmap
...

fig, axes = plt.subplots(2,1,figsize=(8,6), sharex=True)
ax = axes[0]

for c in ['flux','err','full_err']:
    ax.plot(sp.spec_wobs, sp.spec[c], alpha=0.5, label=c)
#ax.plot(sp.spec_wobs, sp.spec['err'], alpha=0.5, label='err')
#ax.plot(sp.spec_wobs, sp.spec['full_err'], alpha=0.5, label='full_err')

ax.set_ylim(-0.1*ymax[1], ymax[1])
ax.plot(sp.spec_wobs, sp.valid*ymax[1]*0.8, alpha=0.5, label='"valid"')
ax.grid()
ax.legend()
ax.set_ylabel(r'$F_\nu\,[\mu\mathrm{Jy}]$')

ax = axes[1]
for c in ['flux','err','full_err']:
    ax.plot(sp.spec_wobs, sp.spec[c]*sp.spec['to_flam'], alpha=0.5, label=c)

ax.set_ylim(-0.1*ymax[1]*5, ymax[1]*5)
ax.set_ylabel(r'$F_\lambda\,[10^{-20}\mathrm{erg/s/cm2/A}]$')
ax.grid()

ax.set_xlabel(r'$\lambda_\mathrm{obs}\,[\mu\mathrm{m}]$')

_ = fig.tight_layout(pad=1)

png

# Show all spectra
fig, axes = plt.subplots(len(xlimits),1,figsize=(8,3*len(xlimits)))

for ax, xlim, ym in zip(axes, xlimits, ymax):
    for i, k in enumerate(sobj):
        sp = sobj[k]
        ax.plot(sp.spec_wobs[sp.valid], sp.spec['flux'][sp.valid], alpha=0.5, label=k.split('_')[1])
    
    ax.set_xlim(*xlim)

    ax.set_ylim(-0.1*ym, ym)
    ax.grid()
    
axes[0].legend()
ax.set_xlabel(r'$\lambda_\mathrm{obs}\,[\mu\mathrm{m}]$')

_ = fig.tight_layout(pad=1)

png

Spectral resolution

sp.spec.info()

<GTable length=1414>
  name    dtype  unit
-------- ------- ----
    wave float64     
    flux float64     
     err float64     
    corr float64     
  escale float64     
full_err float64  uJy
   valid    bool     
       R float64     
 to_flam float64     

fig, ax = plt.subplots(1,1,figsize=(8,3))

for i, k in enumerate(sobj):
    sp = sobj[k]
    ax.plot(sp.spec_wobs, sp.spec['R'], alpha=0.5, label=k.split('_')[1])

ax.legend()
ax.set_ylabel('R')
ax.grid()
ax.semilogy()

ax.set_xlabel(r'$\lambda_\mathrm{obs}\,[\mu\mathrm{m}]$')

_ = fig.tight_layout(pad=1)

png

lw, lr = utils.get_line_wavelengths()

bspl = sp.bspline_array(nspline=21)
bspl.shape

lines = ['Ha','SII-6717','SII-6731']
line_waves = []
for l in lines:
    line_waves += lw[l]



line_um = lw['Ha'][0]*(1+z)/1.e4

fig, axes = plt.subplots(3,1,figsize=(8,8), sharex=True, sharey=False)

counter = 0

for i, k in enumerate(sobj):
    sp = sobj[k]
    if sp.spec_wobs.max() < 3.3:
        continue
    
    ax = axes[counter]
    ax.plot(sp.spec_wobs[sp.valid]/(1+z), sp.spec['flux'][sp.valid], alpha=0.5, label=k.split('_')[1], color='k')
    
    ax.set_yticklabels([])
    ax.set_ylabel(k.split('_')[1])
        
    ### Full line fit
    
    # Dummy spline continuum
    bspl = sp.bspline_array(nspline=21)

    # Generate emission line templates accounting for spectral resolution
    line_in_grating = []
    for lwi in line_waves:
        line_in_grating.append(sp.emission_line(lwi*(1+z)/1.e4,
                                                line_flux=1,
                                                scale_disp=1.3, velocity_sigma=50))

    A = np.vstack([bspl, line_in_grating])
    Ax = A/sp.spec['full_err']
    yx = sp.spec['flux']/sp.spec['full_err']

    lsq = np.linalg.lstsq(Ax[:,sp.valid].T, yx[sp.valid])

    model = A.T.dot(lsq[0])
    ax.plot(sp.spec_wobs/(1+z), model, color='r', lw=3, alpha=0.3)
    
    lwi = np.abs(sp.spec_wobs - line_um) < 0.1
    ym = model[lwi].max()*0.5
    ax.set_ylim(-0.1*ym, ym)

    ax.grid()
    counter += 1
    
ax.set_xlim(line_um/(1+z)-0.01, line_um/(1+z)+0.02)

ax.set_xlabel(r'$\lambda_\mathrm{rest}\,[\mu\mathrm{m}]$')

_ = fig.tight_layout(pad=1)

png

Smooth a full template

Generate a template here, but can also be, e.g. a FSPS or bagpipes model

import eazy.templates

wtempl = np.logspace(3, 4, 4096*4)
ftempl = wtempl*0.

for w in line_waves:
    dv = (wtempl-w)/w*3.e5
    vwidth = 50
    ftempl += 1/np.sqrt(2*np.pi*vwidth**2) * np.exp(-dv**2/2/vwidth**2)

templ = eazy.templates.Template(arrays=(wtempl, ftempl*1.e11))

# Now fold it through the spectrum dispersion

fig, ax = plt.subplots(1,1,figsize=(8,4), sharex=True, sharey=False)

counter = 0

ax.plot(wtempl*(1+z)/1.e4, ftempl, label='intrinsic')

for i, k in enumerate(sobj):
    sp = sobj[k]
    if sp.spec_wobs.max() < 3.3:
        continue

    resamp = sp.resample_eazy_template(templ, z=z, scale_disp=1.3, velocity_sigma=50)
    ax.plot(sp.spec_wobs, resamp, label=k.split('_')[1])
    
ax.legend()
ax.set_xlim(line_um-0.02, line_um+0.12)


(3.3599737502612492, 3.4999737502612494)

png

# Profile the resample function
%timeit resamp = sp.resample_eazy_template(templ, z=z, scale_disp=1.3, velocity_sigma=50)

906 µs ± 9.98 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

Full fitting functions

These are the msaexp functions that were used to fit the reshifts and line fluxes in the full catalog.

for url in urls:
    if 'prism' in url:
        break

_file = os.path.basename(url)
if not os.path.exists(_file):
    ! wget {url}

import astropy.units
_ = msaexp.spectrum.plot_spectrum(url, z=z, plot_unit=astropy.units.microJansky)

# line flux err
# flux x 10^-20 erg/s/cm2
# https://s3.amazonaws.com/msaexp-nirspec/extractions/gds-deep-lr-v1/gds-deep-lr-v1_prism-clear_1210_15157.spec.fits
# z = 4.14871
# Tue Jul 18 21:48:58 2023
             line Hb       76.2 ±      3.7
             line Hg       26.2 ±      6.1
             line Hd       18.2 ±      4.9
         line Ha+NII      263.0 ±      4.2
     line NeIII-3968       26.3 ±      5.7
      line OIII-4959      185.4 ±      5.8
      line OIII-5007      452.2 ±      8.6
      line OIII-4363        1.2 ±      5.8
            line OII      131.9 ±      7.2
      line HeII-4687       -1.5 ±      3.1
     line NeIII-3867       40.2 ±     10.1
       line HeI-3889       12.5 ±      9.5
            line SII       19.7 ±      1.4
       line OII-7325        5.2 ±      1.1
     line ArIII-7138        3.3 ±      1.0
     line ArIII-7753        1.4 ±      0.9
      line SIII-9068        6.9 ±      1.0
      line SIII-9531       20.0 ±      1.5
        line OI-6302        6.1 ±      1.8
            line PaD        2.3 ±      1.1
            line Pa8        7.4 ±      1.4
            line Pa9       -0.3 ±      1.0
           line Pa10        3.6 ±      1.0
       line HeI-5877        8.1 ±      1.7
      line OIII-1663       10.1 ±     12.8
      line CIII-1908       47.5 ±     13.7
      line NIII-1750       17.6 ±     13.1
            line Lya     -107.9 ±     22.9
           line MgII       -7.3 ±      9.2
       line NeV-3346        5.8 ±      6.8
      line NeVI-3426       18.0 ±      6.9
       line HeI-7065        2.9 ±      1.0
       line HeI-8446        0.8 ±      0.9

png

# Fit redshift
_ = msaexp.spectrum.fit_redshift(_file, z0=[z-0.1, z+0.1])

20it [00:00, 88.12it/s]
101it [00:01, 89.43it/s]



# line flux err
# flux x 10^-20 erg/s/cm2
# gds-deep-lr-v1_prism-clear_1210_15157.spec.fits
# z = 4.14755
# Tue Jul 18 21:49:01 2023
             line Hb       76.9 ±      3.9
             line Hg       27.2 ±      6.3
             line Hd       17.0 ±      5.8
         line Ha+NII      265.8 ±      4.2
     line NeIII-3968       22.9 ±      6.3
      line OIII-4959      174.2 ±      5.9
      line OIII-5007      457.3 ±      8.8
      line OIII-4363        2.2 ±      5.9
            line OII      132.0 ±      7.7
      line HeII-4687       -0.1 ±      3.2
     line NeIII-3867       37.7 ±     10.5
       line HeI-3889       11.9 ±      9.7
            line SII       17.4 ±      1.5
       line OII-7325        4.9 ±      1.1
     line ArIII-7138        2.7 ±      1.1
     line ArIII-7753        0.7 ±      1.0
      line SIII-9068        7.0 ±      1.0
      line SIII-9531       16.5 ±      1.5
        line OI-6302        7.6 ±      1.9
            line PaD        1.6 ±      1.2
            line Pa8       10.8 ±      1.4
            line Pa9        0.5 ±      1.0
           line Pa10        3.1 ±      1.0
       line HeI-5877        8.4 ±      1.8
      line OIII-1663       12.4 ±     13.7
      line CIII-1908       35.9 ±     14.6
      line NIII-1750       11.6 ±     13.8
            line Lya     -104.6 ±     24.1
           line MgII       -0.2 ±     10.7
       line NeV-3346        3.2 ±      8.5
      line NeVI-3426       17.5 ±      7.4
       line HeI-7065        3.0 ±      1.1
       line HeI-8446        1.3 ±      1.0

png

png

TBD

Items under development:

  • Scale spectrum to photometry
  • Fit for emission line widths
]]>Gabriel BrammerDJA Imaging Data Products2023-07-18T15:28:54+00:002023-07-18T15:28:54+00:00https://dawn-cph.github.io/dja/blog/2023/07/18/image-data-products imaging demo jwst

(This page is auto-generated from the Jupyter notebook image-data-products.ipynb.)

Here we summarize the files available for imaging datasets, for example in the v7 data release.

%matplotlib inline
import os
import warnings
warnings.filterwarnings('ignore')

import numpy as np
import matplotlib.pyplot as plt
import scipy.ndimage as nd

import astropy.io.fits as pyfits
import astropy.units as u

import sep

import grizli
from grizli import utils
print(f'grizli version: {grizli.__version__}')

BASE_URL = 'https://s3.amazonaws.com/grizli-v2/JwstMosaics/v7/'

grizli version: 1.10.dev3+g341a999

File extensions

Generally, for a given root and filter combination, the following files are available:

  • {root}-{filter}_drc_sci.fits.gz: Science image
  • {root}-{filter}_drc_wht.fits.gz: Inverse variance weight image (sky + readnoise)
  • {root}-{filter}_drc_exp.fits.gz: Exposure-time map
  • {root}-{filter}_wcs.csv: Table summarizing individual exposures that contribute to the mosaic

Notes

  1. All mosaics are created with the legacy drizzlepac.adrizzle.do_driz drizzle implementation that works interchangeably with JWST and HST.
    • grizli generates WCS for each exposure that follow the SIP-WCS convention and that match the newer gwcs JWST wcs at the level of 1e-4 pixels or better
  2. All NIRCam LW, NIRISS (and HST) mosaics are created with 40 mas pixels
  3. Most fields have 20 mas pixels for the NIRCam SW images that exacly subsample the LW grid 2x2.
    • The very large primer-cosmos and primer-uds SW mosaics have 40 mas pixels
  4. All sci mosaics have intensity units of 10 nJy / pix, corresponding to an AB magnitude zeropoint 28.9. This has the slightly desirable property that the image pixel values are not too different from unity.
    • These are not the same as the surface brightness units of the JWST pipeline!
  5. The exp exposure time images have units of seconds rounded to the nearest integer
    • Subsampled to 4x4 of the parent mosaic to keep the file sizes small
    • The exp images are created directly from the footprints of the constituent exposures and don’t account for masked pixels within an exposure.
# Example
root = 'smacs0723-grizli-v7.0'

filter = 'f444w-clear'

# Open the files directly from the web

img = {}

print('# File shape')

for ext in ['sci','wht','exp']:
    _file = f'{root}-{filter}_drc_{ext}.fits.gz'
    img[ext] = pyfits.open(os.path.join(BASE_URL, _file))
    print(f'{_file} : {img[ext][0].data.shape}')

# File shape
smacs0723-grizli-v7.0-f444w-clear_drc_sci.fits.gz : (12000, 12000)
smacs0723-grizli-v7.0-f444w-clear_drc_wht.fits.gz : (12000, 12000)
smacs0723-grizli-v7.0-f444w-clear_drc_exp.fits.gz : (3000, 3000)

# Files have a single PrimaryHDU
img['sci'].info()

Filename: /Users/gbrammer/.astropy/cache/download/url/b0380671ce11dec1c5653485f66f705c/contents
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      93   (12000, 12000)   float32

Primary sci header

img['sci'][0].header

SIMPLE  =                    T / conforms to FITS standard                      
BITPIX  =                  -32 / array data type                                
NAXIS   =                    2 / number of array dimensions                     
NAXIS1  =                12000                                                  
NAXIS2  =                12000                                                  
WCSAXES =                    2 / Number of coordinate axes                      
CRPIX1  =               4591.5 / Pixel coordinate of reference point            
CRPIX2  =               6515.5 / Pixel coordinate of reference point            
CD1_1   = -1.1111111111111E-05 / Coordinate transformation matrix element       
CD2_2   =  1.1111111111111E-05 / Coordinate transformation matrix element       
CDELT1  =                  1.0 / [deg] Coordinate increment at reference point  
CDELT2  =                  1.0 / [deg] Coordinate increment at reference point  
CUNIT1  = 'deg'                / Units of coordinate increment and value        
CUNIT2  = 'deg'                / Units of coordinate increment and value        
CTYPE1  = 'RA---TAN'           / Right ascension, gnomonic projection           
CTYPE2  = 'DEC--TAN'           / Declination, gnomonic projection               
CRVAL1  =            110.83403 / [deg] Coordinate value at reference point      
CRVAL2  =            -73.45429 / [deg] Coordinate value at reference point      
LONPOLE =                180.0 / [deg] Native longitude of celestial pole       
LATPOLE =            -73.45429 / [deg] Native latitude of celestial pole        
MJDREF  =                  0.0 / [d] MJD of fiducial time                       
DATE-OBS= '2022-06-07'         / ISO-8601 time of observation                   
MJD-OBS =              59737.0 / [d] MJD of observation                         
RADESYS = 'ICRS'               / Equatorial coordinate system                   
CD1_2   =                  0.0                                                  
CD2_1   =                  0.0                                                  
DRIZKERN= 'square  '           / Drizzle kernel                                 
DRIZPIXF=                 0.75 / Drizzle pixfrac                                
EXPTIME =    15074.42400000001                                                  
NDRIZIM =                   18                                                  
PIXFRAC =                 0.75                                                  
KERNEL  = 'square  '                                                            
OKBITS  =                    4 / FLT bits treated as valid                      
PHOTSCAL=    1.001401962747847 / Scale factor applied                           
GRIZLIV = '1.8.16.dev12+g86ad0c1' / Grizli code version                         
WHTTYPE = 'jwst    '           / Exposure weighting strategy                    
RNPERC  =                   99 / VAR_RNOISE clip percentile for JWST            
TELESCOP= 'JWST    '                                                            
FILTER  = 'F444W   '                                                            
PUPIL   = 'CLEAR   '                                                            
DETECTOR= 'NRCALONG'                                                            
INSTRUME= 'NIRCAM  '                                                            
PHOTFLAM= 1.54184756289340E-22                                                  
PHOTPLAM=    44036.71097714713                                                  
PHOTFNU =                1E-08                                                  
EXPSTART=    59737.22120032604                                                  
EXPEND  =    59737.23101751157                                                  
TIME-OBS= '05:18:31.708'                                                        
UPDA_CTX= 'jwst_0995.pmap'                                                      
CRDS_CTX= 'jwst_1041.pmap'                                                      
R_DISTOR= 'jwst_nircam_distortion_0141.asdf'                                    
R_PHOTOM= 'jwst_nircam_photom_0111.fits'                                        
R_FLAT  = 'jwst_nircam_flat_0574.fits'                                          
PHOTMJSR=   0.3925000131130219                                                  
PIXAR_SR=             9.31E-14                                                  
FLT00001= 'jw02736001001_02105_00001_nrcalong_rate.fits'                        
WHT00001=      14335.236328125 / Median weight of exposure 1                    
FLT00002= 'jw02736001001_02105_00001_nrcblong_rate.fits'                        
WHT00002=     14818.2607421875 / Median weight of exposure 2                    
FLT00003= 'jw02736001001_02105_00002_nrcalong_rate.fits'                        
WHT00003=      14245.970703125 / Median weight of exposure 3                    
FLT00004= 'jw02736001001_02105_00002_nrcblong_rate.fits'                        
WHT00004=     14709.4404296875 / Median weight of exposure 4                    
FLT00005= 'jw02736001001_02105_00003_nrcalong_rate.fits'                        
WHT00005=     14335.7177734375 / Median weight of exposure 5                    
FLT00006= 'jw02736001001_02105_00003_nrcblong_rate.fits'                        
WHT00006=     14799.9052734375 / Median weight of exposure 6                    
FLT00007= 'jw02736001001_02105_00004_nrcalong_rate.fits'                        
WHT00007=       14333.94921875 / Median weight of exposure 7                    
FLT00008= 'jw02736001001_02105_00004_nrcblong_rate.fits'                        
WHT00008=       14851.24609375 / Median weight of exposure 8                    
FLT00009= 'jw02736001001_02105_00005_nrcalong_rate.fits'                        
WHT00009=     14321.4130859375 / Median weight of exposure 9                    
FLT00010= 'jw02736001001_02105_00005_nrcblong_rate.fits'                        
WHT00010=      14816.029296875 / Median weight of exposure 10                   
FLT00011= 'jw02736001001_02105_00006_nrcalong_rate.fits'                        
WHT00011=     14384.9931640625 / Median weight of exposure 11                   
FLT00012= 'jw02736001001_02105_00006_nrcblong_rate.fits'                        
WHT00012=      14822.169921875 / Median weight of exposure 12                   
FLT00013= 'jw02736001001_02105_00007_nrcalong_rate.fits'                        
WHT00013=       14322.65234375 / Median weight of exposure 13                   
FLT00014= 'jw02736001001_02105_00007_nrcblong_rate.fits'                        
WHT00014=     14808.3759765625 / Median weight of exposure 14                   
FLT00015= 'jw02736001001_02105_00008_nrcalong_rate.fits'                        
WHT00015=     14282.8134765625 / Median weight of exposure 15                   
FLT00016= 'jw02736001001_02105_00008_nrcblong_rate.fits'                        
WHT00016=     14759.8623046875 / Median weight of exposure 16                   
FLT00017= 'jw02736001001_02105_00009_nrcalong_rate.fits'                        
WHT00017=     14353.7353515625 / Median weight of exposure 17                   
FLT00018= 'jw02736001001_02105_00009_nrcblong_rate.fits'                        
WHT00018=     14783.2705078125 / Median weight of exposure 18                   
OPHOTFNU= 9.30775449348276E-08 / Original PHOTFNU before scaling                
BUNIT   = '10.0*nanoJansky'                                                     

# Images have units of 10 nJy / pix
for k in ('FILTER','PHOTFNU','PHOTPLAM','BUNIT'):
    print(f"{k:>8}: {img['sci'][0].header[k]}")

  FILTER: F444W
 PHOTFNU: 1e-08
PHOTPLAM: 44036.71097714713
   BUNIT: 10.0*nanoJansky

Primary exp header

img['exp'][0].header

SIMPLE  =                    T / conforms to FITS standard                      
BITPIX  =                   32 / array data type                                
NAXIS   =                    2 / number of array dimensions                     
NAXIS1  =                 3000                                                  
NAXIS2  =                 3000                                                  
WCSAXES =                    2 / Number of coordinate axes                      
CRPIX1  =             1147.875 / Pixel coordinate of reference point            
CRPIX2  =             1628.875 / Pixel coordinate of reference point            
CD1_1   = -4.4444444444444E-05 / Coordinate transformation matrix element       
CD2_2   =  4.4444444444444E-05 / Coordinate transformation matrix element       
CDELT1  =                  1.0 / [deg] Coordinate increment at reference point  
CDELT2  =                  1.0 / [deg] Coordinate increment at reference point  
CUNIT1  = 'deg'                / Units of coordinate increment and value        
CUNIT2  = 'deg'                / Units of coordinate increment and value        
CTYPE1  = 'RA---TAN'           / Right ascension, gnomonic projection           
CTYPE2  = 'DEC--TAN'           / Declination, gnomonic projection               
CRVAL1  =            110.83403 / [deg] Coordinate value at reference point      
CRVAL2  =            -73.45429 / [deg] Coordinate value at reference point      
LONPOLE =                180.0 / [deg] Native longitude of celestial pole       
LATPOLE =            -73.45429 / [deg] Native latitude of celestial pole        
MJDREF  =                  0.0 / [d] MJD of fiducial time                       
DATE-OBS= '2022-06-07'         / ISO-8601 time of observation                   
MJD-OBS =              59737.0 / [d] MJD of observation                         
RADESYS = 'ICRS'               / Equatorial coordinate system                   
CD1_2   =                  0.0                                                  
CD2_1   =                  0.0                                                  
DRIZKERN= 'square  '           / Drizzle kernel                                 
DRIZPIXF=                 0.75 / Drizzle pixfrac                                
EXPTIME =    15074.42400000001                                                  
NDRIZIM =                   18                                                  
PIXFRAC =                 0.75                                                  
KERNEL  = 'square  '                                                            
OKBITS  =                    4 / FLT bits treated as valid                      
PHOTSCAL=    1.001401962747847 / Scale factor applied                           
GRIZLIV = '1.8.16.dev12+g86ad0c1' / Grizli code version                         
WHTTYPE = 'jwst    '           / Exposure weighting strategy                    
RNPERC  =                   99 / VAR_RNOISE clip percentile for JWST            
TELESCOP= 'JWST    '                                                            
FILTER  = 'F444W   '                                                            
PUPIL   = 'CLEAR   '                                                            
DETECTOR= 'NRCALONG'                                                            
INSTRUME= 'NIRCAM  '                                                            
PHOTFLAM= 1.54184756289340E-22                                                  
PHOTPLAM=    44036.71097714713                                                  
PHOTFNU =                1E-08                                                  
EXPSTART=    59737.22120032604                                                  
EXPEND  =    59737.23101751157                                                  
TIME-OBS= '05:18:31.708'                                                        
UPDA_CTX= 'jwst_0995.pmap'                                                      
CRDS_CTX= 'jwst_1041.pmap'                                                      
R_DISTOR= 'jwst_nircam_distortion_0141.asdf'                                    
R_PHOTOM= 'jwst_nircam_photom_0111.fits'                                        
R_FLAT  = 'jwst_nircam_flat_0574.fits'                                          
PHOTMJSR=   0.3925000131130219                                                  
PIXAR_SR=             9.31E-14                                                  
FLT00001= 'jw02736001001_02105_00001_nrcalong_rate.fits'                        
WHT00001=      14335.236328125 / Median weight of exposure 1                    
FLT00002= 'jw02736001001_02105_00001_nrcblong_rate.fits'                        
WHT00002=     14818.2607421875 / Median weight of exposure 2                    
FLT00003= 'jw02736001001_02105_00002_nrcalong_rate.fits'                        
WHT00003=      14245.970703125 / Median weight of exposure 3                    
FLT00004= 'jw02736001001_02105_00002_nrcblong_rate.fits'                        
WHT00004=     14709.4404296875 / Median weight of exposure 4                    
FLT00005= 'jw02736001001_02105_00003_nrcalong_rate.fits'                        
WHT00005=     14335.7177734375 / Median weight of exposure 5                    
FLT00006= 'jw02736001001_02105_00003_nrcblong_rate.fits'                        
WHT00006=     14799.9052734375 / Median weight of exposure 6                    
FLT00007= 'jw02736001001_02105_00004_nrcalong_rate.fits'                        
WHT00007=       14333.94921875 / Median weight of exposure 7                    
FLT00008= 'jw02736001001_02105_00004_nrcblong_rate.fits'                        
WHT00008=       14851.24609375 / Median weight of exposure 8                    
FLT00009= 'jw02736001001_02105_00005_nrcalong_rate.fits'                        
WHT00009=     14321.4130859375 / Median weight of exposure 9                    
FLT00010= 'jw02736001001_02105_00005_nrcblong_rate.fits'                        
WHT00010=      14816.029296875 / Median weight of exposure 10                   
FLT00011= 'jw02736001001_02105_00006_nrcalong_rate.fits'                        
WHT00011=     14384.9931640625 / Median weight of exposure 11                   
FLT00012= 'jw02736001001_02105_00006_nrcblong_rate.fits'                        
WHT00012=      14822.169921875 / Median weight of exposure 12                   
FLT00013= 'jw02736001001_02105_00007_nrcalong_rate.fits'                        
WHT00013=       14322.65234375 / Median weight of exposure 13                   
FLT00014= 'jw02736001001_02105_00007_nrcblong_rate.fits'                        
WHT00014=     14808.3759765625 / Median weight of exposure 14                   
FLT00015= 'jw02736001001_02105_00008_nrcalong_rate.fits'                        
WHT00015=     14282.8134765625 / Median weight of exposure 15                   
FLT00016= 'jw02736001001_02105_00008_nrcblong_rate.fits'                        
WHT00016=     14759.8623046875 / Median weight of exposure 16                   
FLT00017= 'jw02736001001_02105_00009_nrcalong_rate.fits'                        
WHT00017=     14353.7353515625 / Median weight of exposure 17                   
FLT00018= 'jw02736001001_02105_00009_nrcblong_rate.fits'                        
WHT00018=     14783.2705078125 / Median weight of exposure 18                   
OPHOTFNU= 9.30775449348276E-08 / Original PHOTFNU before scaling                
BUNIT   = 'second  '                                                            
SAMPLE  =                    4 / Sampling factor                                
NXORIG  =                12000                                                  
NYORIG  =                12000                                                  
MOSPSCL =  0.03999999999999958 / Mosaic pixel scale arcsec                      
ORIGPSCL=   0.0629361212228063 / Original detector pixel scale arcsec           
DNTOEPS =    58.62417855098175 / Inverse flux conversion back to e per second   
BSCALE  =                    1                                                  
BZERO   =           2147483648

WCS log

The wcs.csv files contain the full SIP header of each exposure that contributes to the mosaic, along with some epoch information.

_file = f'{root}-{filter}_wcs.csv'
wcs = utils.read_catalog(os.path.join(BASE_URL, _file))
print(wcs.colnames)

['file', 'ext', 'exptime', 'wcsaxes', 'crpix1', 'crpix2', 'cd1_1', 'cd1_2', 'cd2_1', 'cd2_2', 'cdelt1', 'cdelt2', 'cunit1', 'cunit2', 'ctype1', 'ctype2', 'crval1', 'crval2', 'lonpole', 'latpole', 'wcsname', 'mjdref', 'date-beg', 'mjd-beg', 'date-avg', 'mjd-avg', 'date-end', 'mjd-end', 'xposure', 'telapse', 'obsgeo-x', 'obsgeo-y', 'obsgeo-z', 'radesys', 'velosys', 'a_order', 'a_0_2', 'a_0_3', 'a_0_4', 'a_0_5', 'a_1_1', 'a_1_2', 'a_1_3', 'a_1_4', 'a_2_0', 'a_2_1', 'a_2_2', 'a_2_3', 'a_3_0', 'a_3_1', 'a_3_2', 'a_4_0', 'a_4_1', 'a_5_0', 'b_order', 'b_0_2', 'b_0_3', 'b_0_4', 'b_0_5', 'b_1_1', 'b_1_2', 'b_1_3', 'b_1_4', 'b_2_0', 'b_2_1', 'b_2_2', 'b_2_3', 'b_3_0', 'b_3_1', 'b_3_2', 'b_4_0', 'b_4_1', 'b_5_0', 'naxis', 'naxis1', 'naxis2', 'sipcrpx1', 'sipcrpx2']

# First few lines
wcs['file','ext','exptime','mjd-avg','date-avg','crpix1','crpix2','crval1','crval2'][:4]
GTable length=4
fileextexptimemjd-avgdate-avgcrpix1crpix2crval1crval2
str44int64float64float64str23float64float64float64float64
jw02736001001_02105_00001_nrcalong_rate.fits1837.46859737.2261089192022-06-07T05:25:35.8111024.6471024.66110.68612332528-73.481395298838
jw02736001001_02105_00001_nrcblong_rate.fits1837.46859737.2261052152022-06-07T05:25:35.4911024.4891024.662110.8276309259-73.453986741388
jw02736001001_02105_00002_nrcalong_rate.fits1837.46859737.2367955742022-06-07T05:40:59.1381024.6471024.66110.68291892635-73.47999884627
jw02736001001_02105_00002_nrcblong_rate.fits1837.46859737.236791872022-06-07T05:40:58.8181024.4891024.662110.82443032455-73.452590822835

Compare the WHT and EXP images


exts = ['sci','wht','exp']

fig, axes = plt.subplots(2,len(exts),figsize=(3*len(exts),6))

for j, ext in enumerate(exts):
    msk = img[ext][0].data != 0
    wmax = np.nanpercentile(img[ext][0].data[msk], 95)
    for i in [0,1]:
        axes[i][j].imshow(img[ext][0].data, vmin=0, vmax=wmax, origin='lower', cmap='magma')
        axes[i][j].grid()
        if i == 0:
            axes[i][j].set_title(ext)
        
xy = 2600, 6100, 256

for j, p in enumerate([0,0,1]):
    axes[0][j].set_xlim(*(xy[0] + np.array([-1,1])*xy[2])/4**p)
    axes[0][j].set_ylim(*(xy[1] + np.array([-1,1])*xy[2])/4**p)
    
fig.tight_layout(pad=1)

png

Make a full variance image including the Poisson component from the sources themselves

The mosaics are created by weighting each input exposure by a factor like 1/wht = VAR_RNOISE + median(VAR_POISSON) from the JWST exposure files. The first term incorporates pixel-to-pixel variations resulting from pixels where some fraction of the reads may have been masked as saturated or affected by cosmic rays. The second term effectively provides the noise from the sky background, but without including the Poisson term for individual sources.

Certain applications like photometry or morphology fitting with galfit may require a variance / sigma image that includes the poisson term from the individual sources. This can be generated from the exp maps as shown below.

# Grow the exposure map to the original frame
full_exp = np.zeros(img['sci'][0].data.shape, dtype=int)
full_exp[2::4,2::4] += img['exp'][0].data*1
full_exp = nd.maximum_filter(full_exp, 4)

img['Full exp'] = pyfits.HDUList([pyfits.PrimaryHDU(data=full_exp)])

# Show the full exposure map
exts = ['wht','Full exp','exp']

fig, axes = plt.subplots(2,len(exts),figsize=(3*len(exts),6))

for j, ext in enumerate(exts):
    msk = img[ext][0].data != 0
    wmax = np.nanpercentile(img[ext][0].data[msk], 95)
    for i in [0,1]:
        axes[i][j].imshow(img[ext][0].data, vmin=0, vmax=wmax, origin='lower', cmap='magma')
        axes[i][j].grid()
        if i == 0:
            axes[i][j].set_title(ext)
        
xy = 2600, 6100, 256

for j, p in enumerate([0,0,1]):
    axes[0][j].set_xlim(*(xy[0] + np.array([-1,1])*xy[2])/4**p)
    axes[0][j].set_ylim(*(xy[1] + np.array([-1,1])*xy[2])/4**p)
    
fig.tight_layout(pad=1)

png

Effective “gain”

To convert the Poisson variance associated with the sci image, write down the multiplicative factors that had been applied to the original count-rate data in the pipeline rate files. PHOTMJSR is the original flux calibration to units of “MJy/sr” and PHOTSCL is any (small) additional photometric term that was included by grizli. The PHOTFNU / OPHOTFNU term accounts for any final scaling of the output mosaic and the ratio of the original and mosaic pixel areas.

# Scale factors
phot_scale = 1 / (PHOTMJSR * PHOTSCAL) * PHOTFNU / OPHOTFNU

# Effective gain e-/DN, including exposure time
effective_gain = phot_scale * exposure_time_map

# Variance in electrons = counts in electrons
var_poisson_elec = sci * effective_gain

# Variance in mosaic DN
var_poisson_dn = var_poisson_elec / effective_gain**2 = sci / effective_gain

header = img['exp'][0].header

# Multiplicative factors that have been applied since the original count-rate images
phot_scale = 1.

for k in ['PHOTMJSR','PHOTSCAL']:
    print(f'{k} {header[k]:.3f}')
    phot_scale /= header[k]

# Unit and pixel area scale factors
if 'OPHOTFNU' in header:
    phot_scale *= header['PHOTFNU'] / header['OPHOTFNU']

# "effective_gain" = electrons per DN of the mosaic
effective_gain = (phot_scale * full_exp)

# Poisson variance in mosaic DN
var_poisson_dn = np.maximum(img['sci'][0].data, 0) / effective_gain

# Original variance from the `wht` image = RNOISE + BACKGROUND
var_wht = 1/img['wht'][0].data

# New total variance
var_total = var_wht + var_poisson_dn
full_wht = 1 / var_total

# Null weights
full_wht[var_total <= 0] = 0

img['Full wht'] = pyfits.HDUList([pyfits.PrimaryHDU(data=full_wht, header=img['wht'][0].header)])

PHOTMJSR 0.393
PHOTSCAL 1.001

# Compare science and weight arrays, where you can now 
# "see" the sources in the Full weight array

exts = ['sci','Full wht','wht']

fig, axes = plt.subplots(2,len(exts),figsize=(3*len(exts),6))

for j, ext in enumerate(exts):
    msk = img[ext][0].data != 0
    wmax = np.nanpercentile(img[ext][0].data[msk], 95)
    for i in [0,1]:
        axes[i][j].imshow(img[ext][0].data, vmin=0, vmax=wmax, origin='lower', cmap='magma')
        axes[i][j].grid()
        if i == 0:
            axes[i][j].set_title(ext)
        
xy = 2600, 6100, 256

for j, p in enumerate([0,0,0]):
    axes[0][j].set_xlim(*(xy[0] + np.array([-1,1])*xy[2])/4**p)
    axes[0][j].set_ylim(*(xy[1] + np.array([-1,1])*xy[2])/4**p)
    
fig.tight_layout(pad=1)

png

]]>Gabriel BrammerCatalog Demo - GOODS-South2023-07-14T23:18:17+00:002023-07-14T23:18:17+00:00https://dawn-cph.github.io/dja/blog/2023/07/14/photometric-catalog-demo imaging catalog gds

(This page is auto-generated from the Jupyter notebook photometric-catalog-demo.ipynb.)

Show how to interact with the DJA/grizli photometric catalogs.

(little explanatory text for the quick demo)

%matplotlib inline

import os
import yaml

import numpy as np
import matplotlib.pyplot as plt

import warnings
warnings.filterwarnings('ignore')

import astropy.io.fits as pyfits

import grizli
import grizli.catalog
from grizli import utils

import eazy

print(f'grizli version: {grizli.__version__}')
print(f'eazy-py version: {eazy.__version__}')


grizli version: 1.10.dev2+g661e5ea
eazy-py version: 0.6.5

Set the field

Currently available:

  • gds = GOODS-South
  • gdn = GOODS-North
  • ceers-full = CEERS EGS
  • abell2744clu = Abell 2744 GLASS + UNCOVER + DD-2756
  • macs0647 = MACS 0647 cluster (Coe et al., GO-1433)
  • rxj2129 = RXJ 2129 cluster (Kelly et al., DD-2767)
  • sunrise = “Sunrise Arc” (WHL0137, Coe et al., GO-2282)
  • smacs0723 = SMACS 0723 cluster (Pontoppidan et al., DD-2736)
  • …
field = 'gds-grizli-v7.0'

url_path = 'https://s3.amazonaws.com/grizli-v2/JwstMosaics/v7'

Raw photometry

NB: All photometry given in fnu flux densities with units of microJansky (AB zeropoint = 23.9).

phot = utils.read_catalog(f'{url_path}/{field}_phot.fits')

Metadata

# General data of the source detection
for i, k in enumerate(phot.meta):
    print(f'{k:>36} = {phot.meta[k]}')
    if i > 70:
        print('...')
        break

                             VERSION = 1.2.1
                             MINAREA = 9
                               CLEAN = True
                             DEBCONT = 0.001
                            DEBTHRSH = 32
                         FILTER_TYPE = conv
                           THRESHOLD = 1.5
                            KRONFACT = 2.5
                               KRON0 = 2.4
                               KRON1 = 3.8
                             MINKRON = 8.750000000000059
                            TOTCFILT = F140W
                            TOTCWAVE = 13922.907
                                  ZP = 28.9
                                PLAM = 13922.907
                                 FNU = 1e-08
                                FLAM = 1.4737148e-20
                              UJY2DN = 99.99395614709495
                            DRZ_FILE = gds-grizli-v7.0-ir_drc_sci.fits.gz
                            WHT_FILE = gds-grizli-v7.0-ir_drc_wht.fits.gz
                            GET_BACK = True
                             BACK_BW = 50
                             BACK_BH = 50
                             BACK_FW = 3
                             BACK_FH = 3
                    BACK_PIXEL_SCALE = 0.04
                           ERR_SCALE = 0.8123676180839539
                            RESCALEW = True
                            APERMASK = True
                                GAIN = 2000.0
                              APER_0 = 9.00000000000006
                              ASEC_0 = 0.36
                              APER_1 = 12.50002500000008
                              ASEC_1 = 0.5000009999999999
                              APER_2 = 17.50000500000012
                              ASEC_2 = 0.7000002000000001
                              APER_3 = 25.00000500000017
                              ASEC_3 = 1.0000002
                CLEARP-F430M_VERSION = 1.2.1
                     CLEARP-F430M_ZP = 28.9
                   CLEARP-F430M_PLAM = 42816.84172704966
                    CLEARP-F430M_FNU = 1e-08
                   CLEARP-F430M_FLAM = 1.63095481504235e-22
                 CLEARP-F430M_uJy2dn = 99.99395614709495
               CLEARP-F430M_DRZ_FILE = gds-grizli-v7.0-clearp-f430m_drc_sci
               CLEARP-F430M_WHT_FILE = gds-grizli-v7.0-clearp-f430m_drc_wht
               CLEARP-F430M_GET_BACK = True
                CLEARP-F430M_BACK_BW = 50
                CLEARP-F430M_BACK_BH = 50
                CLEARP-F430M_BACK_FW = 3
                CLEARP-F430M_BACK_FH = 3
       CLEARP-F430M_BACK_PIXEL_SCALE = 0.04
              CLEARP-F430M_ERR_SCALE = 1.0
               CLEARP-F430M_RESCALEW = True
               CLEARP-F430M_APERMASK = True
                   CLEARP-F430M_GAIN = 2000.0
                 CLEARP-F430M_aper_0 = 9.00000000000006
                 CLEARP-F430M_asec_0 = 0.36
                 CLEARP-F430M_aper_1 = 12.50002500000008
                 CLEARP-F430M_asec_1 = 0.5000009999999999
                 CLEARP-F430M_aper_2 = 17.50000500000012
                 CLEARP-F430M_asec_2 = 0.7000002000000001
                 CLEARP-F430M_aper_3 = 25.00000500000017
                 CLEARP-F430M_asec_3 = 1.0000002
                CLEARP-F480M_VERSION = 1.2.1
                     CLEARP-F480M_ZP = 28.9
                   CLEARP-F480M_PLAM = 48151.90220745452
                    CLEARP-F480M_FNU = 1e-08
                   CLEARP-F480M_FLAM = 1.28956813045049e-22
                 CLEARP-F480M_uJy2dn = 99.99395614709495
               CLEARP-F480M_DRZ_FILE = gds-grizli-v7.0-clearp-f480m_drc_sci
               CLEARP-F480M_WHT_FILE = gds-grizli-v7.0-clearp-f480m_drc_wht
...

Photometric apertures

for i, k in enumerate(phot.meta):
    if k.startswith('APER_'):
        aper_index = k[-1]
        print(f"Aperture index {aper_index}: *diameter* = {phot.meta[k]:4.1f} pixels = {phot.meta[k.replace('APER','ASEC')]:.2f} arcsec")

Aperture index 0: *diameter* =  9.0 pixels = 0.36 arcsec
Aperture index 1: *diameter* = 12.5 pixels = 0.50 arcsec
Aperture index 2: *diameter* = 17.5 pixels = 0.70 arcsec
Aperture index 3: *diameter* = 25.0 pixels = 1.00 arcsec

# Columns for a particular filter + aperture
aper_index = '1'

cols = []

for k in phot.colnames:
    if k.startswith('f444w') & k.endswith(aper_index):
        cols.append(k)
        
phot[cols].info()

<GTable length=52427>
           name             dtype  unit    class     n_bad
-------------------------- ------- ---- ------------ -----
   f444w-clear_flux_aper_1 float64  uJy MaskedColumn   640
f444w-clear_fluxerr_aper_1 float64  uJy MaskedColumn   640
   f444w-clear_flag_aper_1   int16            Column     0
    f444w-clear_bkg_aper_1 float64  uJy MaskedColumn   641
   f444w-clear_mask_aper_1 float64            Column     0

Photometric bands

  • NIRCam filters generally have “clear” in the filter name, which is the element in the pupil wheel.
  • Filters that start with clearp are generally the long-wavelength NIRISS filters.
  • Filters with names that end in wn are the NIRISS versions, e.g., f200wn-clear for NIRISS and f200w-clear for NIRCam
  • HST filters ending in “u” are the WFC3/UVIS versions, e.g., f814wu
count = 0
for k in phot.colnames:
    if k.endswith('_flux_aper_1'):
        count += 1
        print(f"{count:>2} {k.split('_flux')[0]}")

 1 clearp-f430m
 2 clearp-f480m
 3 f090w-clear
 4 f105w
 5 f110w
 6 f115w-clear
 7 f115wn-clear
 8 f125w
 9 f140w
10 f150w-clear
11 f150wn-clear
12 f160w
13 f182m-clear
14 f200w-clear
15 f200wn-clear
16 f210m-clear
17 f277w-clear
18 f335m-clear
19 f336wu
20 f350lpu
21 f356w-clear
22 f410m-clear
23 f430m-clear
24 f435w
25 f444w-clear
26 f460m-clear
27 f475w
28 f480m-clear
29 f606w
30 f606wu
31 f775w
32 f814w
33 f814wu
34 f850lp
35 f850lpu

# Missing data are *masked*

fig, axes = plt.subplots(1,2,figsize=(10,5), sharex=True, sharey=True)
cosd = np.cos(np.nanmedian(phot['dec'])/180*np.pi)

axes[0].scatter(phot['ra'], phot['dec'], c=phot['f444w-clear_flux_aper_1'].mask)
axes[0].set_title('F444W (JADES + FRESCO)')

axes[1].scatter(phot['ra'], phot['dec'], c=phot['f277w-clear_flux_aper_1'].mask)
axes[1].set_title('F277W (JADES)')

axes[0].set_xlim(*axes[0].get_xlim()[::-1])
for ax in axes:
    ax.set_aspect(1./cosd)
    ax.grid()

png

# 5-sigma depth in the D=0.5" aperture
depth = 23.9 - 2.5*np.log10(phot['f444w-clear_flux_aper_1']*5)
fig, ax = plt.subplots(1,1,figsize=(6,5))
ax.set_aspect(1./cosd)
so = np.argsort(depth)
sc = ax.scatter(phot['ra'][so], phot['dec'][so], c=depth[so], vmin=26, vmax=31, alpha=0.5)

ax.set_xlim(*axes[0].get_xlim()[::-1])

cb = plt.colorbar(sc)
cb.set_label('raw depth, D=0.5" aperture')

png

Point sources

fig, ax = plt.subplots(1,1,figsize=(8,5))
in_jades = ~phot['f277w-clear_flux_aper_1'].mask
ax.scatter(phot['mag_auto'], phot['flux_radius'], alpha=0.2, c=in_jades, vmin=0, vmax=2, cmap='viridis')
ax.set_ylim(0,10)
ax.grid()
ax.set_xlim(17, 32)

ax.set_xlabel('mag_auto (detection band)')
ax.set_ylabel('flux_radius')


Text(0, 0.5, 'flux_radius')

png

Catalog with aperture corrections

  • {filter}_corr_{aper} = {filter}_corr_{aper} * flux_auto / flux_{aper} : Aperture corrected to the auto flux in the detection band
  • {filter}_tot_{aper} = {filter}_corr_{aper} * {filter}_tot_corr : Corrected for flux outside of the auto aperture. Not implemented for JWST, where corr = tot
if not os.path.exists(f'{field}-fix.photoz.tar.gz'):
    ! wget {url_path}/{field}-fix.photoz.tar.gz

! tar xzvf {field}-fix.photoz.tar.gz

x gds-grizli-v7.0-fix.eazypy.h5
x gds-grizli-v7.0-fix.eazypy.residuals.001.png
x gds-grizli-v7.0-fix.eazypy.residuals.002.png
x gds-grizli-v7.0-fix.eazypy.residuals.003.png
x gds-grizli-v7.0-fix.eazypy.zout.fits
x gds-grizli-v7.0-fix.eazypy.zphot.param
x gds-grizli-v7.0-fix.eazypy.zphot.translate
x gds-grizli-v7.0-fix.eazypy.zphot.zeropoint
x gds-grizli-v7.0-fix.zhist.png
x gds-grizli-v7.0-fix.zphot_zspec.png
x gds-grizli-v7.0-fix_phot_apcorr.fits

apc = utils.read_catalog(f'{field}-fix_phot_apcorr.fits')

aper_index = '1'

cols = []

for k in apc.colnames:
    if k.startswith('f444w') & k.endswith(aper_index):
        cols.append(k)
        
apc[cols].info()

<GTable length=52427>
        name          dtype  unit    class     n_bad
-------------------- ------- ---- ------------ -----
   f444w_flux_aper_1 float64  uJy MaskedColumn   640
f444w_fluxerr_aper_1 float64  uJy MaskedColumn   640
   f444w_flag_aper_1   int16      MaskedColumn     0
    f444w_bkg_aper_1 float64  uJy MaskedColumn   641
   f444w_mask_aper_1 float64            Column     0
        f444w_corr_1 float64  uJy       Column     0
       f444w_ecorr_1 float64  uJy       Column     0
         f444w_tot_1 float64  uJy       Column     0
        f444w_etot_1 float64  uJy       Column     0

Compare to Skelton 3D-HST

import grizli.catalog
s14 = grizli.catalog.query_tap_catalog(ra=np.nanmedian(phot['ra']), dec=np.nanmedian(phot['dec']),
                                       radius=10,
                                       vizier=True,
                                       db='"J/ApJS/214/24/3dhstall"')
len(s14)

idx, dr, dx, dy = utils.GTable(s14).match_to_catalog_sky(apc, get_2d_offset=True)

fig, ax = plt.subplots(1,1,figsize=(5,5))

ax.scatter(dx, dy, alpha=0.1)

has_match = dr.value < 0.3
ra_offset = np.nanmedian((apc['ra'] - s14['RAJ2000'][idx])[has_match])
dec_offset = np.nanmedian((apc['dec'] - s14['DEJ2000'][idx])[has_match])

s14['ra'] += ra_offset
s14['dec'] += dec_offset

idx, dr, dx, dy = utils.GTable(s14).match_to_catalog_sky(apc, get_2d_offset=True)
ax.scatter(dx, dy, alpha=0.1)

has_match = dr.value < 0.2

ax.set_xlim(-0.5, 0.5)
ax.set_ylim(*ax.get_xlim())
ax.grid()

ax.set_xlabel(r'$\Delta RA, arcsec')
ax.set_ylabel(r'$\Delta Dec, arcsec')


Query "J/ApJS/214/24/3dhstall" from VizieR TAP server
Launched query: 'SELECT TOP 1000000 * FROM "J/ApJS/214/24/3dhstall" WHERE RAJ2000 > 53.05300162385859 AND RAJ2000 < 53.24140686277677 AND DEJ2000 > -27.87878381298912 AND DEJ2000 < -27.712117146322456 '
------>http
host = tapvizier.u-strasbg.fr:80
context = /TAPVizieR/tap/sync
Content-type = application/x-www-form-urlencoded
200 200
[('date', 'Fri, 14 Jul 2023 22:54:49 GMT'), ('server', 'Apache/2.4.41 (Ubuntu) mod_jk/1.2.46 OpenSSL/1.1.1f'), ('vary', 'Accept-Encoding'), ('access-control-allow-origin', '*'), ('access-control-allow-credentials', 'true'), ('transfer-encoding', 'chunked'), ('content-type', 'application/x-votable+xml; serialization=TABLEDATA;charset=UTF-8')]
Retrieving sync. results...
Saving results to: sync_20230715005449.xml
Query finished.





Text(0, 0.5, '$\\Delta Dec, arcsec')

png

mag_s14_aper = 25 - 2.5*np.log10(s14['F160Wap'])
mag_jw_d0p7 = 23.9 - 2.5*np.log10(phot['f160w_flux_aper_2'])

mag_s14_tot = 25 - 2.5*np.log10(s14['F160W'])
mag_jw_tot = 23.9 - 2.5*np.log10(apc['f160w_tot_1'])

delta_mag_aper = mag_jw_d0p7 - mag_s14_aper[idx]
delta_mag_tot = mag_jw_tot - mag_s14_tot[idx]

# jh_jw = -2.5*np.log10(apc['f814w_tot_1']*0.847/apc['f160w_tot_1'])
# jh_s14 = -2.5*np.log10(s14['F814W']/s14['F160W'])

jh_jw = -2.5*np.log10(apc['f125w_tot_1']/apc['f160w_tot_1'])
jh_s14 = -2.5*np.log10(s14['F125W']/s14['F160W'])

delta_color = jh_jw - jh_s14[idx]

fig, axes = plt.subplots(3,1,figsize=(8,6), sharex=True, sharey=True)

axes[0].scatter(mag_jw_tot[has_match], delta_mag_aper[has_match], alpha=0.1)
axes[0].set_ylabel(r'$\Delta$mag F160W' + '\nD=0.7" aperture')

axes[1].scatter(mag_jw_tot[has_match], delta_mag_tot[has_match], vmax=1.5, alpha=0.1)
axes[1].set_ylabel(r'$\Delta$mag F160W' + '\n total corrected')

axes[2].scatter(mag_jw_tot[has_match], delta_color[has_match], alpha=0.1)
axes[2].set_ylabel(r'$\Delta$color' + '\nF125W - F160W')

axes[2].set_xlabel('mag, HST F160W')

for ax in axes:
    ax.set_xlim(18, 29)
    ax.set_ylim(-1,1)
    ax.grid()

png

Photometric redshifts

import eazy.hdf5

if not os.path.exists('templates'):
    eazy.symlink_eazy_inputs()
    
root = f'{field}-fix'

self = eazy.hdf5.initialize_from_hdf5(h5file=root+'.eazypy.h5')
self.fit_phoenix_stars()

zout = utils.read_catalog(root+'.eazypy.zout.fits')
self.cat = utils.read_catalog(root+'_phot_apcorr.fits')
cat = self.cat

Read default param file: /Users/gbrammer/miniconda3/envs/py39jw/lib/python3.9/site-packages/eazy/data/zphot.param.default
CATALOG_FILE is a table
   >>> NOBJ = 52427
f090w_tot_1 f090w_etot_1 (363): jwst_nircam_f090w
f105w_tot_1 f105w_etot_1 (202): hst/wfc3/IR/f105w.dat
f110w_tot_1 f110w_etot_1 (241): hst/wfc3/IR/f110w.dat
f115w_tot_1 f115w_etot_1 (364): jwst_nircam_f115w
f115wn_tot_1 f115wn_etot_1 (309): niriss-f115w
f125w_tot_1 f125w_etot_1 (203): hst/wfc3/IR/f125w.dat
f140w_tot_1 f140w_etot_1 (204): hst/wfc3/IR/f140w.dat
f150w_tot_1 f150w_etot_1 (365): jwst_nircam_f150w
f150wn_tot_1 f150wn_etot_1 (310): niriss-f150w
f160w_tot_1 f160w_etot_1 (205): hst/wfc3/IR/f160w.dat
f182m_tot_1 f182m_etot_1 (370): jwst_nircam_f182m
f200w_tot_1 f200w_etot_1 (366): jwst_nircam_f200w
f200wn_tot_1 f200wn_etot_1 (311): niriss-f200w
f210m_tot_1 f210m_etot_1 (371): jwst_nircam_f210m
f277w_tot_1 f277w_etot_1 (375): jwst_nircam_f277w
f335m_tot_1 f335m_etot_1 (381): jwst_nircam_f335m
f356w_tot_1 f356w_etot_1 (376): jwst_nircam_f356w
f410m_tot_1 f410m_etot_1 (383): jwst_nircam_f410m
f430m_tot_1 f430m_etot_1 (384): jwst_nircam_f430m
f435w_tot_1 f435w_etot_1 (233): hst/ACS_update_sep07/wfc_f435w_t81.dat
f444w_tot_1 f444w_etot_1 (377): jwst_nircam_f444w
f460m_tot_1 f460m_etot_1 (385): jwst_nircam_f460m
f475w_tot_1 f475w_etot_1 (234): hst/ACS_update_sep07/wfc_f475w_t81.dat
f480m_tot_1 f480m_etot_1 (386): jwst_nircam_f480m
f606w_tot_1 f606w_etot_1 (236): hst/ACS_update_sep07/wfc_f606w_t81.dat
f606wu_tot_1 f606wu_etot_1 (214): hst/wfc3/UVIS/f606w.dat
f775w_tot_1 f775w_etot_1 (238): hst/ACS_update_sep07/wfc_f775w_t81.dat
f814w_tot_1 f814w_etot_1 (239): hst/ACS_update_sep07/wfc_f814w_t81.dat
f814wu_tot_1 f814wu_etot_1 (217): hst/wfc3/UVIS/f814w.dat
f850lp_tot_1 f850lp_etot_1 (240): hst/ACS_update_sep07/wfc_f850lp_t81.dat
Set sys_err = 0.05 (positive=True)
Read PRIOR_FILE:  templates/prior_F160W_TAO.dat
Template grid: templates/sfhz/agn_blue_sfhz_13.param (this may take some time)
TemplateGrid: user-provided tempfilt_data
Process templates: 0.237 s


294it [00:02, 104.84it/s]


h5: read corr_sfhz_13_bin0_av0.01.fits
h5: read corr_sfhz_13_bin0_av0.25.fits
h5: read corr_sfhz_13_bin0_av0.50.fits
h5: read corr_sfhz_13_bin0_av1.00.fits
h5: read corr_sfhz_13_bin1_av0.01.fits
h5: read corr_sfhz_13_bin1_av0.25.fits
h5: read corr_sfhz_13_bin1_av0.50.fits
h5: read corr_sfhz_13_bin1_av1.00.fits
h5: read corr_sfhz_13_bin2_av0.01.fits
h5: read corr_sfhz_13_bin2_av0.50.fits
h5: read corr_sfhz_13_bin2_av1.00.fits
h5: read corr_sfhz_13_bin3_av0.01.fits
h5: read corr_sfhz_13_bin3_av0.50.fits
h5: read fsps_4590.fits
h5: read j0647agn+torus.fits
fit_best: 2.5 s (n_proc=5,  NOBJ=51074)
phoenix_templates: ./bt-settl_t400-7000_g4.5.fits

Adjusted zeropoints

The iterated eazy “zeropoint” adjustments are used as a crude PSF-matching correction for the simple aperture photometry catalogs. That is, all photometry is done on the native images and corrected to “total” using a single correction derived in the detection band. The encircled energy (for point sources) will be different for the different instruments / filters, and the eazy zeropoint adjustments are initialized with corrections that would be appropriate to put point sources on a common scale. Corrections to that are then derived based on the photo-z fits to the full catalog.

print('# ix filt f_number zp')
for i in np.argsort(self.lc):
    print(f"{i:2} {self.flux_columns[i].split('_')[0]:12} {self.f_numbers[i]:3} {self.zp[i]:.3f}")

# ix filt f_number zp
19 f435w        233 1.067
22 f475w        234 1.091
25 f606wu       214 0.929
24 f606w        236 0.911
26 f775w        238 0.865
27 f814w        239 0.847
28 f814wu       217 1.000
 0 f090w        363 0.931
29 f850lp       240 0.875
 1 f105w        202 0.932
 4 f115wn       309 0.872
 2 f110w        241 0.942
 3 f115w        364 0.876
 5 f125w        203 0.948
 6 f140w        204 0.965
 8 f150wn       310 0.878
 7 f150w        365 0.871
 9 f160w        205 0.983
10 f182m        370 0.909
11 f200w        366 0.903
12 f200wn       311 0.882
13 f210m        371 0.936
14 f277w        375 1.000
15 f335m        381 1.052
16 f356w        376 1.077
17 f410m        383 1.114
18 f430m        384 1.138
20 f444w        377 1.148
21 f460m        385 1.175
23 f480m        386 1.205

fig = eazy.utils.zphot_zspec(zout['z_phot'][has_match], zout['z_spec'][has_match], zmax=8)
fig.axes[0].set_title(f'JWST {field}')

fig = eazy.utils.zphot_zspec(s14['zpk'][idx][has_match], zout['z_spec'][has_match], zmax=8)
fig.axes[0].set_title(f'Skelton (2014)')

Text(0.5, 1.0, 'Skelton (2014)')

png

png

Plot some SEDs

# e.g., GOODS-S-9209 from Carnall et al. https://arxiv.org/pdf/2301.11413.pdf
ra, dec = 53.1082274, -27.8252019

dr = np.sqrt((zout['ra'] - ra)**2 + (zout['dec'] - dec)**2)

self.cat['z_spec'][np.argmin(dr)] = 4.6582

cat_id = zout['id'][np.argmin(dr)]

_ = self.show_fit(cat_id, zr=[3,7])

png

def quick_cutout(resp, sy=2, pl=2, size=1.5, scl=3, filters='f115w-clear,f277w-clear,f444w-clear'):
    """
    Make a cutout figure with the grizli cutout server
    """
    from PIL import Image
    import requests
    from io import BytesIO

    #rd = ds9.get('pan icrs').replace(' ',',')
    
    if isinstance(resp, int):
        id = resp
        ix = np.where(self.cat['id'] == id)[0][0]
    else:
        ix = resp['ix']
        id = resp['id']

    rd = f"{self.RA[ix]:.6f},{self.DEC[ix]:.6f}"
    
    
    print(rd)
    print(f"https://s3.amazonaws.com/grizli-v2/ClusterTiles/Map/gds/jwst.html?coord={rd}&zoom=6")

    url=f"https://grizli-cutout.herokuapp.com/thumb?coord={rd}&all_filters=True&size={size}&scl={scl}&asinh=True&filters={filters}&rgb_scl=1.0,0.95,1.2&pl={pl}"

    response = requests.get(url)
    img = Image.open(BytesIO(response.content))
    img.apply_transparency()

    fig, ax = plt.subplots(1,1, figsize=(sy*4, sy+0.2), sharex=True, sharey=True)
    data = np.array(img)
    black = data.max(axis=2) == 0
    for ioff in range(-2,3):
        black &= np.roll(black, ioff, axis=0)


    for i in range(3):
        data[:,:,i][black] = 255
    
    ax.imshow(data, interpolation='Nearest', origin='upper')
    
    #ax.text(0.5, 0.01, 'nrc', color='r', fontsize=8, ha='center',va='bottom',transform=ax.transAxes)        
    ax.set_xticklabels([])
    ax.set_yticklabels([])
    ax.axis('off')
    ax.text(0.05, -0.01, f"{root}   {self.cat['id'][ix]}   ({rd.replace(',','  ')})    z_phot={self.zbest[ix]:.3f}    logM={np.log10(zout['mass'][ix]):.1f}".replace('z_phot', r'$z_\mathrm{phot}$'),
            ha='left', va='top', transform=ax.transAxes, fontsize=7)
    
    ax.text(0.95, -0.01, filters.replace('-clear','').replace(',', ' '), ha='right', va='top', 
            fontsize=7, color='k', transform=ax.transAxes)
    
    fig.tight_layout(pad=0.2)
        
    return id, fig, img

if 1:
    id, fig, img = quick_cutout(_[1], pl=2, scl=4, sy=3, filters='f775w,f182m-clear,f444w-clear')

# id, fig, img = quick_cutout(_[1], pl=2, scl=5, sy=3)


53.108211,-27.825183
https://s3.amazonaws.com/grizli-v2/ClusterTiles/Map/gds/jwst.html?coord=53.108211,-27.825183&zoom=6

png

# Curtis Lake - JADES
src = utils.read_catalog('https://raw.githubusercontent.com/dawn-cph/jwst-sources/main/jwst-sources.csv')
idx, dr = zout.match_to_catalog_sky(src)

ecl = src['author'] == 'Emma Curtis-Lake'
ids_jades = zout['id'][idx][ecl]
z_jades = src['zspec'][ecl]

i = -1

self.cat['z_spec'][idx] = src['zspec']

for i in np.argsort(z_jades):
    id = ids_jades[i]
    _ = self.show_fit(id, show_fnu=True, maglim=(32,27))
    # id, fig, img = quick_cutout(_[1], pl=2, scl=8, sy=3, size=1, filters='f090w-clear,f115w-clear,f150w-clear,f200w-clear,f277w-clear,f356w-clear,f444w-clear')
    id, fig, img = quick_cutout(_[1], scl=10, size=1, filters='f115w-clear,f150w-clear,f200w-clear')


53.158837,-27.773500
https://s3.amazonaws.com/grizli-v2/ClusterTiles/Map/gds/jwst.html?coord=53.158837,-27.773500&zoom=6
53.164768,-27.774627
https://s3.amazonaws.com/grizli-v2/ClusterTiles/Map/gds/jwst.html?coord=53.164768,-27.774627&zoom=6
53.166346,-27.821558
https://s3.amazonaws.com/grizli-v2/ClusterTiles/Map/gds/jwst.html?coord=53.166346,-27.821558&zoom=6
53.149886,-27.776504
https://s3.amazonaws.com/grizli-v2/ClusterTiles/Map/gds/jwst.html?coord=53.149886,-27.776504&zoom=6

png

png

png

png

png

png

png

png

# More filters
id, fig, img = quick_cutout(_[1], pl=2, scl=8, sy=3, size=1, filters='f090w-clear,f115w-clear,f150w-clear,f200w-clear,f277w-clear,f356w-clear,f444w-clear')

53.149886,-27.776504
https://s3.amazonaws.com/grizli-v2/ClusterTiles/Map/gds/jwst.html?coord=53.149886,-27.776504&zoom=6

png

]]>Gabriel Brammer