Reading in and writing out observations

Loading SDFITS files

Classes and functions for importing SDFITS files

SDFITSLoad

Load generic SDFITS files - Not typically used directly. Sub-class for specific telescope SDFITS flavors.

class dysh.fits.sdfitsload.SDFITSLoad(filename, source=None, hdu=None, **kwargs)

Bases: object

Generic Container for a bintable(s) from selected HDU(s) for a single SDFITS file. For multiple SDFITS files, see GBTFITSLoad.

Parameters:

filenamestr

input file name

sourcestr

target source to select from input file. Default: all sources

hduint or list

Header Data Unit to select from input file. Default: all HDUs

Attributes:

bintable

The list of bintables

filename

The input SDFITS filename

Methods

binheader()	The list of bintable headers
create_index([hdu])	Create the index of the SDFITS file.
fix_meta(meta)	Do any repair to the meta/header for peculariaties in definitions from a particular observatory The passed-in dictionary will be repaired in place.
getrow(i[, bintable])	Get a FITS_record from the input bintable
getspec(i[, bintable])	Get a row (record) as a Spectrum
index([hdu, bintable])	Return The index table.
info()	Return the HDUList info()
load([hdu])	Load the bintable for given hdu.
naxis(naxis, bintable)	The NAXISn value of the input bintable.
nchan(bintable)	The number of channels per row of the input bintable.
nintegrations(bintable[, source])	The number of integrations on a given source divided by the number of polarizations
npol(bintable)	The number of polarizations present in the input bintable.
nrows(bintable)	The number of rows of the input bintable
rawspectra(bintable)	Get the raw (unprocessed) spectra from the input bintable.
rawspectrum(i[, bintable])	Get a single raw (unprocessed) spectrum from the input bintable.
scans(bintable)	The number of scans resent in the input bintable.
sources(bintable)	The number of sources present in the input bintable.
summary()	Print a summary of each record of the data
udata(key[, bintable])	The unique list of values of a given header keyword
ushow(key[, bintable])	Print the unique list of values of a given header keyword
velocity_convention(veldef, velframe)	Compute the velocity convention string use for velocity conversions, given the VELDEF and VELFRAME values.
write(fileobj[, rows, bintable, ...])	Write the SDFITSLoad to a new file, potentially sub-selecting rows or bintables.

add_primary_hdu

add_primary_hdu()

binheader()

The list of bintable headers

property bintable

The list of bintables

create_index(hdu=None)

Create the index of the SDFITS file.

Parameters:

hduint or list

Header Data Unit to select from input file. Default: all HDUs

property filename

The input SDFITS filename

fix_meta(meta)

Do any repair to the meta/header for peculariaties in definitions from a particular observatory The passed-in dictionary will be repaired in place. At minimum this method must populate meta[‘VELDEF’] and meta[‘VELFRAME’]

Parameters:

metadict

The header of the Spectrum to be fixed, corresponding to the meta attribute of the Spectrum.

getrow(i, bintable=0)

Get a FITS_record from the input bintable

Parameters:

iint

The record (row) index to retrieve

bintableint

The index of the bintable attribute

Returns:

rowFITS_record

The i-th record of the input bintable

getspec(i, bintable=0)

Get a row (record) as a Spectrum

index(hdu=None, bintable=None)

Return The index table.

Parameters:

hduint or list

Header Data Unit to select from the index. Default: all HDUs

bintableint

The index of the bintable attribute, None means all bintables

Returns:

index~pandas.DataFrame

The index of this SDFITS file

info()

Return the HDUList info()

load(hdu=None, **kwargs)

Load the bintable for given hdu. Note mmHg and UTC are unrecognized units. mmHg is in astropy.units.cds but UTC is just wrong.

Parameters:

hduint or list

Header Data Unit to select from input file. Default: all HDUs

naxis(naxis, bintable)

The NAXISn value of the input bintable.

Parameters:

naxisint

The NAXIS whose length is requested

bintableint

The index of the bintable attribute

Returns:

naxisthe length of the NAXIS

nchan(bintable)

The number of channels per row of the input bintable. Assumes all rows have same length.

Parameters:

bintableint

The index of the bintable attribute

Returns:

nchanint

Number channels in the first spectrum of the input bintbale

nintegrations(bintable, source=None)

The number of integrations on a given source divided by the number of polarizations

Parameters:

bintableint

The index of the bintable attribute

source: str

The source name (OBJECT keyword) or None for all sources. Default: None

Returns:

nintegrationsthe number of integrations

npol(bintable)

The number of polarizations present in the input bintable.

Parameters:

bintableint

The index of the bintable attribute

Returns:

npol: int

Number of polarizations as given by CRVAL4 FITS header keyword.

nrows(bintable)

The number of rows of the input bintable

Parameters:

bintableint

The index of the bintable attribute

Returns:

nrowsint

Number of rows, i.e., the length of the input bintable

rawspectra(bintable)

Get the raw (unprocessed) spectra from the input bintable.

Parameters:

bintableint

The index of the bintable attribute

Returns:

rawspectra~numpy.ndarray

The DATA column of the input bintable

rawspectrum(i, bintable=0)

Get a single raw (unprocessed) spectrum from the input bintable.

Parameters:

iint

The row index to retrieve.

bintableint or None

The index of the bintable attribute. If None, the underlying bintable is computed from i

Returns:

rawspectrum~numpy.ndarray

The i-th row of DATA column of the input bintable

scans(bintable)

The number of scans resent in the input bintable. TODO: move this to GBTFISLoad?

Parameters:

bintableint

The index of the bintable attribute

Returns:

scans: int

Number of scans as given by SCAN FITS header keyword.

sources(bintable)

The number of sources present in the input bintable.

Parameters:

bintableint

The index of the bintable attribute

Returns:

sources: int

Number of sources as given by OBJECT FITS header keyword.

summary()

Print a summary of each record of the data

udata(key, bintable=None)

The unique list of values of a given header keyword

Parameters:

keystr

The keyword to retrieve

bintableint

The index of the bintable attribute, None means all bintables

Returns:

udatalist

The unique set of values for the input keyword.

ushow(key, bintable=None)

Print the unique list of values of a given header keyword

Parameters:

keystr

The keyword to retrieve

bintableint

The index of the bintable attribute, None means all bintables

velocity_convention(veldef, velframe)

Compute the velocity convention string use for velocity conversions, given the VELDEF and VELFRAME values. Return value must be a recognized string of Spectrum1D, one of {“doppler_relativistic”, “doppler_optical”, “doppler_radio”} Sub-classes should implement, because different observatories use VELDEF and VELFRAME inconsistently. This base class method hard-coded to return “doppler_radio.”

Parameters:

veldefstr

The velocity definition string (VELDEF FITS keyword)

velframestr

The velocity frame string (VELFRAME FITS keyword)

write(fileobj, rows=None, bintable=None, output_verify='exception', overwrite=False, checksum=False)

Write the SDFITSLoad to a new file, potentially sub-selecting rows or bintables.

Parameters:

fileobjstr, file-like or pathlib.Path

File to write to. If a file object, must be opened in a writeable mode.

rows: int or list-like

Range of rows in the bintable(s) to write out. e.g. 0, [14,25,32]. Default: None, meaning all rows Note: Currently rows, if given, must be contained in a single bintable and bintable must be given

bintableint

The index of the bintable attribute or None for all bintables. Default: None

output_verifystr

Output verification option. Must be one of "fix", "silentfix", "ignore", "warn", or "exception". May also be any combination of "fix" or "silentfix" with "+ignore", +warn, or +exception" (e.g. ``"fix+warn"). See https://docs.astropy.org/en/latest/io/fits/api/verification.html for more info

overwritebool, optional

If True, overwrite the output file if it exists. Raises an OSError if False and the output file exists. Default is False.

checksumbool

When True adds both DATASUM and CHECKSUM cards to the headers of all HDU’s written to the file.

GBTFITSLoad

Load SDFITS files produced by the Green Bank Telescope

class dysh.fits.gbtfitsload.GBTFITSLoad(fileobj, source=None, hdu=None, **kwargs)

Bases: SDFITSLoad

GBT-specific container to reprensent one or more SDFITS files

Parameters:

fileobjstr or pathlib.Path

File to read or directory path. If a directory, all FITS files within will be read in.

sourcestr

target source to select from input file(s). Default: all sources

hduint or list

Header Data Unit to select from input file. Default: all HDUs

Attributes:

bintable

The list of bintables

filename

The input SDFITS filename

Methods

binheader()	The list of bintable headers
calonoff_rows([scans, bintable, fitsindex])	Get individual scan row numbers sorted by whether the calibration (diode) was on or off, and selected by ifnum,plnum, fdnum,subref,bintable.
create_index([hdu])	Create the index of the SDFITS file.
fix_meta(meta)	Do any repair to the meta/header for peculariaties in definitions from a particular observatory The passed-in dictionary will be repaired in place.
getps([scans, bintable])	Get the rows that contain position-switched data.
getrow(i[, bintable])	Get a FITS_record from the input bintable
getspec(i[, bintable])	Get a row (record) as a Spectrum
gettp(scan[, sig, cal, bintable])	Get a total power scan, optionally calibrating it.
index([hdu, bintable, fitsindex])	Return The index table
info()	Return the HDUList info()
load([hdu])	Load the bintable for given hdu.
naxis(naxis, bintable)	The NAXISn value of the input bintable.
nchan(bintable)	The number of channels per row of the input bintable.
nintegrations(bintable[, source])	The number of integrations on a given source divided by the number of polarizations
npol(bintable)	The number of polarizations present in the input bintable.
nrows(bintable)	The number of rows of the input bintable
onoff_rows([scans, ifnum, plnum, bintable, ...])	Get individual ON/OFF (position switch) scan row numbers selected by ifnum,plnum, bintable.
onoff_scan_list([scans, ifnum, plnum, ...])	Get the scans for position-switch data sorted
rawspectra(bintable, fitsindex)	Get the raw (unprocessed) spectra from the input bintable.
rawspectrum(i[, bintable, fitsindex])	Get a single raw (unprocessed) spectrum from the input bintable.
scan_rows(scans[, ifnum, plnum, bintable, ...])	Get scan rows selected by ifnum,plnum, bintable.
scans(bintable)	The number of scans resent in the input bintable.
select(key, value, df)	Select data where key=value.
sources(bintable)	The number of sources present in the input bintable.
subbeamnod(scan[, bintable])	Get a subbeam nod power scan, optionally calibrating it.
summary([scans, verbose, show_index])	Create a summary list of the input dataset.
udata(key[, bintable])	The unique list of values of a given header keyword
ushow(key[, bintable])	Print the unique list of values of a given header keyword
velocity_convention(veldef)	Given the GBT VELDEF FITS string return the specutils velocity convention, e.g., "doppler_radio"
velocity_frame(veldef)	Given the GBT VELDEF FITS string return the velocity frame, e.g., "heliocentric".
write(fileobj[, rows, bintable, ...])	Write the SDFITSLoad to a new file, potentially sub-selecting rows or bintables.
write_scans(fileobj, scans[, output_verify, ...])	Write specific scans of the GBTFITSLoad to a new file.

add_primary_hdu
files
select_onoff
select_scans

calonoff_rows(scans=None, bintable=None, fitsindex=0, **kwargs)

Get individual scan row numbers sorted by whether the calibration (diode) was on or off, and selected by ifnum,plnum, fdnum,subref,bintable.

Parameters:

scansint or list-like

The scan numbers to find the rows of

ifnumint

the IF index

plnumint

the polarization index

fdnumint

the feed index

subrefint

the subreflector state (-1,0,1)

bintableint

the index for BINTABLE containing the scans

fitsindex: int

the index of the FITS file contained in this GBTFITSLoad. Default:0

Returns:

rowsdict

A dictionary with keys ‘ON’ and ‘OFF’ giving the row indices of CALON and CALOFF data for the input scan(s)

files()

getps(scans=None, bintable=None, **kwargs)

Get the rows that contain position-switched data. These include ONs and OFFs.

kwargs: plnum, feed, ifnum, integration, calibrate=T/F, average=T/F, tsys, weights

Parameters:

scansint or 2-tuple

Single scan number or list of scan numbers to use. Default: all scans. Scan numbers can be Ons or Offs

weights: str

‘equal’ or ‘tsys’ to indicate equal weighting or tsys weighting to use in time averaging. Default: ‘tsys’

Returns:

dataScanBlock

A ScanBlock containing one or more PSScan

gettp(scan, sig=None, cal=None, bintable=None, **kwargs)

Get a total power scan, optionally calibrating it.

Parameters:

scan: int

scan number

sigbool or None

True to use only integrations where signal state is True, False to use reference state (signal state is False). None to use all integrations.

cal: bool or None

True to use only integrations where calibration (diode) is on, False if off. None to use all integrations regardless calibration state. The system temperature will be calculated from both states regardless of the value of this variable.

bintableint

the index for BINTABLE in sdfits containing the scans

calibrate: bool

whether or not to calibrate the data. If True, the data will be (calon - caloff)*0.5, otherwise it will be SDFITS row data. Default:True

weights: str

‘equal’ or ‘tsys’ to indicate equal weighting or tsys weighting to use in time averaging. Default: ‘tsys’

scan args - ifnum, plnum, fdnum, subref

Returns:

dataScanBlock

A ScanBlock containing one or more TPScan

index(hdu=None, bintable=None, fitsindex=None)

Return The index table

Parameters:

hduint or list

Header Data Unit to select from the index. Default: all HDUs

bintableint

The index of the bintable attribute, None means all bintables

fitsindex: int

The index of the FITS file contained in this GBTFITSLoad. Default:None meaning return one index over all files.

Returns:

index~pandas.DataFrame

The index of this GBTFITSLoad

info()

Return the HDUList info()

onoff_rows(scans=None, ifnum=0, plnum=0, bintable=None, fitsindex=0)

Get individual ON/OFF (position switch) scan row numbers selected by ifnum,plnum, bintable.

Parameters:

scansint or list-like

The scan numbers to find the rows of

ifnumint

the IF index

plnumint

the polarization index

bintableint

the index for BINTABLE in sdfits containing the scans

fitsindex: int

the index of the FITS file contained in this GBTFITSLoad. Default:0

Returns:

rowsdict

A dictionary with keys ‘ON’ and ‘OFF’ giving the row indices of the ON and OFF data for the input scan(s)

onoff_scan_list(scans=None, ifnum=0, plnum=0, bintable=None, fitsindex=0)

Get the scans for position-switch data sorted

by ON and OFF state

Parameters:

scansint or list-like

The scan numbers to find the rows of

ifnumint

the IF index

plnumint

the polarization index

bintableint

the index for BINTABLE containing the scans

fitsindex: int

the index of the FITS file contained in this GBTFITSLoad. Default:0

Returns:

rowsdict

A dictionary with keys ‘ON’ and ‘OFF’ giving the scan numbers of ON and OFF data for the input scan(s)

rawspectra(bintable, fitsindex)

Get the raw (unprocessed) spectra from the input bintable.

Parameters:

bintableint

The index of the bintable attribute

fitsindex: int

the index of the FITS file contained in this GBTFITSLoad. Default:0

Returns:

rawspectra~numpy.ndarray

The DATA column of the input bintable

rawspectrum(i, bintable=0, fitsindex=0)

Get a single raw (unprocessed) spectrum from the input bintable.

Parameters:

iint

The row index to retrieve.

bintableint or None

The index of the bintable attribute. If None, the underlying bintable is computed from i

fitsindex: int

the index of the FITS file contained in this GBTFITSLoad. Default:0

Returns:

rawspectrum~numpy.ndarray

The i-th row of DATA column of the input bintable

scan_rows(scans, ifnum=0, plnum=0, bintable=None, fitsindex=0)

Get scan rows selected by ifnum,plnum, bintable.

Parameters:

scansint or list-like

The scan numbers to find the rows of

ifnumint

the IF index

plnumint

the polarization index

bintableint

the index for BINTABLE in sdfits containing the scans. Default:None

fitsindex: int

the index of the FITS file contained in this GBTFITSLoad. Default:0

Returns:

rowslist

Lists of the rows in each bintable that contain the scans. Index of rows is the bintable index number

select(key, value, df)

Select data where key=value.

Parameters:

keystr

The key value (SDFITS column name)

valueany

The value to match

dfDataFrame

The DataFrame to search

Returns:

dfDataFrame

The subselected DataFrame

select_onoff(df)

select_scans(scans, df)

subbeamnod(scan, bintable=None, **kwargs)

Get a subbeam nod power scan, optionally calibrating it.

Parameters:

scan: int

scan number

method: str

Method to use when processing. One of ‘cycle’ or ‘scan’. ‘cycle’ is more accurate and averages data in each SUBREF_STATE cycle. ‘scan’ reproduces GBTIDL’s snodka function which has been shown to be less accurate. Default:’cycle’

sigbool

True to indicate if this is the signal scan, False if reference

cal: bool

True if calibration (diode) is on, False if off.

bintableint

the index for BINTABLE in sdfits containing the scans, None means use all bintables

calibrate: bool

whether or not to calibrate the data. If True, the data will be (calon - caloff)*0.5, otherwise it will be SDFITS row data. Default:True

weights: str

‘equal’ or ‘tsys’ to indicate equal weighting or tsys weighting to use in time averaging. Default: ‘tsys’

scan args - ifnum, fdnum, subref (plnum depends on fdnum)

Returns:

dataScanBlock

A ScanBlock object containing the data

summary(scans=None, verbose=False, show_index=True)

Create a summary list of the input dataset. If verbose=False (default), some numeric data (e.g., RESTFREQ, AZIMUTH, ELEVATIO) are averaged over the records with the same scan number.

Parameters:

scansint or 2-tuple

The scan(s) to use. A 2-tuple represents (beginning, ending) scans. Default: show all scans

verbose: bool

If True, list every record, otherwise return a compact summary

show_index: bool

If True, show the DataFrame index column. Default: False

Returns:

summary - DataFrame

Summary of the data as a DataFrame.

velocity_convention(veldef)

Given the GBT VELDEF FITS string return the specutils velocity convention, e.g., “doppler_radio”

Parameters:

veldefstr

The FITS header VELDEF string

Returns:

conventionstr

The velocity convention

velocity_frame(veldef)

Given the GBT VELDEF FITS string return the velocity frame, e.g., “heliocentric”.

Parameters:

veldefstr

The FITS header VELDEF string

Returns:

frame: str

The velocity frame

write_scans(fileobj, scans, output_verify='exception', overwrite=False, checksum=False)

Write specific scans of the GBTFITSLoad to a new file. TBD: How does this work for multiple files??

Parameters:

fileobjstr, file-like or pathlib.Path

File to write to. If a file object, must be opened in a writeable mode.

scans: int or list-like

Range of scans to write out. e.g. 0, [14,25,32].

output_verifystr

Output verification option. Must be one of "fix", "silentfix", "ignore", "warn", or "exception". May also be any combination of "fix" or "silentfix" with "+ignore", +warn, or +exception" (e.g. ``"fix+warn"). See https://docs.astropy.org/en/latest/io/fits/api/verification.html for more info

overwritebool, optional

If True, overwrite the output file if it exists. Raises an OSError if False and the output file exists. Default is False.

checksumbool

When True adds both DATASUM and CHECKSUM cards to the headers of all HDU’s written to the file.