{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Chapter 2\n", "\n", "This is the second in a series of notebooks related to astronomy data.\n", "\n", "As a running example, we are replicating parts of the analysis in a recent paper, \"[Off the beaten path: Gaia reveals GD-1 stars outside of the main stream](https://arxiv.org/abs/1805.00425)\" by Adrian M. Price-Whelan and Ana Bonaca.\n", "\n", "In the first notebook, we wrote ADQL queries and used them to select and download data from the Gaia server.\n", "\n", "In this notebook, we'll pick up where we left off and write a query to select stars from the region of the sky where we expect GD-1 to be." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Outline\n", "\n", "We'll start with an example that does a \"cone search\"; that is, it selects stars that appear in a circular region of the sky.\n", "\n", "Then, to select stars in the vicinity of GD-1, we'll:\n", "\n", "* Use `Quantity` objects to represent measurements with units.\n", "\n", "* Use the `Gala` library to convert coordinates from one frame to another.\n", "\n", "* Use the ADQL keywords `POLYGON`, `CONTAINS`, and `POINT` to select stars that fall within a polygonal region.\n", "\n", "* Submit a query and download the results.\n", "\n", "* Store the results in a FITS file.\n", "\n", "After completing this lesson, you should be able to\n", "\n", "* Use Python string formatting to compose more complex ADQL queries.\n", "\n", "* Work with coordinates and other quantities that have units.\n", "\n", "* Download the results of a query and store them in a file." ] }, { "cell_type": "markdown", "metadata": { "tags": [ "remove-cell" ] }, "source": [ "## Installing libraries\n", "\n", "If you are running this notebook on Colab, you can run the following cell to install Astroquery and the other libraries we'll use.\n", "\n", "If you are running this notebook on your own computer, you might have to install these libraries yourself. See the instructions in the preface." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "tags": [ "remove-cell" ] }, "outputs": [], "source": [ "# If we're running on Colab, install libraries\n", "\n", "import sys\n", "IN_COLAB = 'google.colab' in sys.modules\n", "\n", "if IN_COLAB:\n", " !pip install astroquery astro-gala pyia" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Selecting a region" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One of the most common ways to restrict a query is to select stars in a particular region of the sky.\n", "\n", "For example, here's a query from the [Gaia archive documentation](https://gea.esac.esa.int/archive-help/adql/examples/index.html) that selects \"all the objects ... in a circular region centered at (266.41683, -29.00781) with a search radius of 5 arcmin (0.08333 deg).\"" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "query = \"\"\"\n", "SELECT \n", "TOP 10 source_id\n", "FROM gaiadr2.gaia_source\n", "WHERE 1=CONTAINS(\n", " POINT(ra, dec),\n", " CIRCLE(266.41683, -29.00781, 0.08333333))\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This query uses three keywords that are specific to ADQL (not SQL):\n", "\n", "* `POINT`: a location in [ICRS coordinates](https://en.wikipedia.org/wiki/International_Celestial_Reference_System), specified in degrees of right ascension and declination.\n", "\n", "* `CIRCLE`: a circle where the first two values are the coordinates of the center and the third is the radius in degrees.\n", "\n", "* `CONTAINS`: a function that returns `1` if a `POINT` is contained in a shape and `0` otherwise.\n", "\n", "Here is the [documentation of `CONTAINS`](http://www.ivoa.net/documents/ADQL/20180112/PR-ADQL-2.1-20180112.html#tth_sEc4.2.12).\n", "\n", "A query like this is called a cone search because it selects stars in a cone.\n", "\n", "Here's how we run it." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Created TAP+ (v1.2.1) - Connection:\n", "\tHost: gea.esac.esa.int\n", "\tUse HTTPS: True\n", "\tPort: 443\n", "\tSSL Port: 443\n", "Created TAP+ (v1.2.1) - Connection:\n", "\tHost: geadata.esac.esa.int\n", "\tUse HTTPS: True\n", "\tPort: 443\n", "\tSSL Port: 443\n" ] }, { "data": { "text/html": [ "Table length=10\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
source_id
int64
4057468321929794432
4057468287575835392
4057482027171038976
4057470349160630656
4057470039924301696
4057469868125641984
4057468351995073024
4057469661959554560
4057470520960672640
4057470555320409600
" ], "text/plain": [ "\n", " source_id \n", " int64 \n", "-------------------\n", "4057468321929794432\n", "4057468287575835392\n", "4057482027171038976\n", "4057470349160630656\n", "4057470039924301696\n", "4057469868125641984\n", "4057468351995073024\n", "4057469661959554560\n", "4057470520960672640\n", "4057470555320409600" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from astroquery.gaia import Gaia\n", "\n", "job = Gaia.launch_job(query)\n", "result = job.get_results()\n", "result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Exercise\n", "\n", "When you are debugging queries like this, you can use `TOP` to limit the size of the results, but then you still don't know how big the results will be.\n", "\n", "An alternative is to use `COUNT`, which asks for the number of rows that would be selected, but it does not return them.\n", "\n", "In the previous query, replace `TOP 10 source_id` with `COUNT(source_id)` and run the query again. How many stars has Gaia identified in the cone we searched?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting GD-1 Data\n", "\n", "From the Price-Whelan and Bonaca paper, we will try to reproduce Figure 1, which includes this representation of stars likely to belong to GD-1:\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Along the axis of right ascension ($\\phi_1$) the figure extends from -100 to 20 degrees.\n", "\n", "Along the axis of declination ($\\phi_2$) the figure extends from about -8 to 4 degrees.\n", "\n", "Ideally, we would select all stars from this rectangle, but there are more than 10 million of them, so\n", "\n", "* That would be difficult to work with,\n", "\n", "* As anonymous users, we are limited to 3 million rows in a single query, and\n", "\n", "* While we are developing and testing code, it will be faster to work with a smaller dataset.\n", "\n", "So we'll start by selecting stars in a smaller rectangle, from -55 to -45 degrees right ascension and -8 to 4 degrees of declination.\n", "\n", "But first we let's see how to represent quantities with units like degrees." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Working with coordinates\n", "\n", "Coordinates are physical quantities, which means that they have two parts, a value and a unit.\n", "\n", "For example, the coordinate $30^{\\circ}$ has value 30 and its units are degrees.\n", "\n", "Until recently, most scientific computation was done with values only; units were left out of the program altogether, [often with disastrous results](https://en.wikipedia.org/wiki/Mars_Climate_Orbiter#Cause_of_failure).\n", "\n", "Astropy provides tools for including units explicitly in computations, which makes it possible to detect errors before they cause disasters.\n", "\n", "To use Astropy units, we import them like this:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import astropy.units as u\n", "\n", "u" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`u` is an object that contains most common units and all SI units.\n", "\n", "You can use `dir` to list them, but you should also [read the documentation](https://docs.astropy.org/en/stable/units/)." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['A',\n", " 'AA',\n", " 'AB',\n", " 'ABflux',\n", " 'ABmag',\n", " 'AU',\n", " 'Angstrom',\n", " 'B',\n", " 'Ba',\n", " 'Barye',\n", " 'Bi',\n", " 'Biot',\n", " 'Bol',\n", " 'Bq',\n", " 'C',\n", " 'Celsius',\n", " 'Ci',\n", " 'CompositeUnit',\n", " 'D',\n", " 'Da',\n", " 'Dalton',\n", " 'Debye',\n", " 'Decibel',\n", " 'DecibelUnit',\n", " 'Dex',\n", " 'DexUnit',\n", " 'EA',\n", " 'EAU',\n", " 'EB',\n", " 'EBa',\n", " 'EC',\n", " 'ED',\n", " 'EF',\n", " 'EG',\n", " 'EGal',\n", " 'EH',\n", " 'EHz',\n", " 'EJ',\n", " 'EJy',\n", " 'EK',\n", " 'EL',\n", " 'EN',\n", " 'EOhm',\n", " 'EP',\n", " 'EPa',\n", " 'ER',\n", " 'ERy',\n", " 'ES',\n", " 'ESt',\n", " 'ET',\n", " 'EV',\n", " 'EW',\n", " 'EWb',\n", " 'Ea',\n", " 'Eadu',\n", " 'Earcmin',\n", " 'Earcsec',\n", " 'Eau',\n", " 'Eb',\n", " 'Ebarn',\n", " 'Ebeam',\n", " 'Ebin',\n", " 'Ebit',\n", " 'Ebyte',\n", " 'Ecd',\n", " 'Echan',\n", " 'Ecount',\n", " 'Ect',\n", " 'Ed',\n", " 'Edeg',\n", " 'Edyn',\n", " 'EeV',\n", " 'Eerg',\n", " 'Eg',\n", " 'Eh',\n", " 'EiB',\n", " 'Eib',\n", " 'Eibit',\n", " 'Eibyte',\n", " 'Ek',\n", " 'El',\n", " 'Elm',\n", " 'Elx',\n", " 'Elyr',\n", " 'Em',\n", " 'Emag',\n", " 'Emin',\n", " 'Emol',\n", " 'Eohm',\n", " 'Epc',\n", " 'Eph',\n", " 'Ephoton',\n", " 'Epix',\n", " 'Epixel',\n", " 'Erad',\n", " 'Es',\n", " 'Esr',\n", " 'Eu',\n", " 'Evox',\n", " 'Evoxel',\n", " 'Eyr',\n", " 'F',\n", " 'Farad',\n", " 'Fr',\n", " 'Franklin',\n", " 'FunctionQuantity',\n", " 'FunctionUnitBase',\n", " 'G',\n", " 'GA',\n", " 'GAU',\n", " 'GB',\n", " 'GBa',\n", " 'GC',\n", " 'GD',\n", " 'GF',\n", " 'GG',\n", " 'GGal',\n", " 'GH',\n", " 'GHz',\n", " 'GJ',\n", " 'GJy',\n", " 'GK',\n", " 'GL',\n", " 'GN',\n", " 'GOhm',\n", " 'GP',\n", " 'GPa',\n", " 'GR',\n", " 'GRy',\n", " 'GS',\n", " 'GSt',\n", " 'GT',\n", " 'GV',\n", " 'GW',\n", " 'GWb',\n", " 'Ga',\n", " 'Gadu',\n", " 'Gal',\n", " 'Garcmin',\n", " 'Garcsec',\n", " 'Gau',\n", " 'Gauss',\n", " 'Gb',\n", " 'Gbarn',\n", " 'Gbeam',\n", " 'Gbin',\n", " 'Gbit',\n", " 'Gbyte',\n", " 'Gcd',\n", " 'Gchan',\n", " 'Gcount',\n", " 'Gct',\n", " 'Gd',\n", " 'Gdeg',\n", " 'Gdyn',\n", " 'GeV',\n", " 'Gerg',\n", " 'Gg',\n", " 'Gh',\n", " 'GiB',\n", " 'Gib',\n", " 'Gibit',\n", " 'Gibyte',\n", " 'Gk',\n", " 'Gl',\n", " 'Glm',\n", " 'Glx',\n", " 'Glyr',\n", " 'Gm',\n", " 'Gmag',\n", " 'Gmin',\n", " 'Gmol',\n", " 'Gohm',\n", " 'Gpc',\n", " 'Gph',\n", " 'Gphoton',\n", " 'Gpix',\n", " 'Gpixel',\n", " 'Grad',\n", " 'Gs',\n", " 'Gsr',\n", " 'Gu',\n", " 'Gvox',\n", " 'Gvoxel',\n", " 'Gyr',\n", " 'H',\n", " 'Henry',\n", " 'Hertz',\n", " 'Hz',\n", " 'IrreducibleUnit',\n", " 'J',\n", " 'Jansky',\n", " 'Joule',\n", " 'Jy',\n", " 'K',\n", " 'Kayser',\n", " 'Kelvin',\n", " 'KiB',\n", " 'Kib',\n", " 'Kibit',\n", " 'Kibyte',\n", " 'L',\n", " 'L_bol',\n", " 'L_sun',\n", " 'LogQuantity',\n", " 'LogUnit',\n", " 'Lsun',\n", " 'MA',\n", " 'MAU',\n", " 'MB',\n", " 'MBa',\n", " 'MC',\n", " 'MD',\n", " 'MF',\n", " 'MG',\n", " 'MGal',\n", " 'MH',\n", " 'MHz',\n", " 'MJ',\n", " 'MJy',\n", " 'MK',\n", " 'ML',\n", " 'MN',\n", " 'MOhm',\n", " 'MP',\n", " 'MPa',\n", " 'MR',\n", " 'MRy',\n", " 'MS',\n", " 'MSt',\n", " 'MT',\n", " 'MV',\n", " 'MW',\n", " 'MWb',\n", " 'M_bol',\n", " 'M_e',\n", " 'M_earth',\n", " 'M_jup',\n", " 'M_jupiter',\n", " 'M_p',\n", " 'M_sun',\n", " 'Ma',\n", " 'Madu',\n", " 'MagUnit',\n", " 'Magnitude',\n", " 'Marcmin',\n", " 'Marcsec',\n", " 'Mau',\n", " 'Mb',\n", " 'Mbarn',\n", " 'Mbeam',\n", " 'Mbin',\n", " 'Mbit',\n", " 'Mbyte',\n", " 'Mcd',\n", " 'Mchan',\n", " 'Mcount',\n", " 'Mct',\n", " 'Md',\n", " 'Mdeg',\n", " 'Mdyn',\n", " 'MeV',\n", " 'Mearth',\n", " 'Merg',\n", " 'Mg',\n", " 'Mh',\n", " 'MiB',\n", " 'Mib',\n", " 'Mibit',\n", " 'Mibyte',\n", " 'Mjup',\n", " 'Mjupiter',\n", " 'Mk',\n", " 'Ml',\n", " 'Mlm',\n", " 'Mlx',\n", " 'Mlyr',\n", " 'Mm',\n", " 'Mmag',\n", " 'Mmin',\n", " 'Mmol',\n", " 'Mohm',\n", " 'Mpc',\n", " 'Mph',\n", " 'Mphoton',\n", " 'Mpix',\n", " 'Mpixel',\n", " 'Mrad',\n", " 'Ms',\n", " 'Msr',\n", " 'Msun',\n", " 'Mu',\n", " 'Mvox',\n", " 'Mvoxel',\n", " 'Myr',\n", " 'N',\n", " 'NamedUnit',\n", " 'Newton',\n", " 'Ohm',\n", " 'P',\n", " 'PA',\n", " 'PAU',\n", " 'PB',\n", " 'PBa',\n", " 'PC',\n", " 'PD',\n", " 'PF',\n", " 'PG',\n", " 'PGal',\n", " 'PH',\n", " 'PHz',\n", " 'PJ',\n", " 'PJy',\n", " 'PK',\n", " 'PL',\n", " 'PN',\n", " 'POhm',\n", " 'PP',\n", " 'PPa',\n", " 'PR',\n", " 'PRy',\n", " 'PS',\n", " 'PSt',\n", " 'PT',\n", " 'PV',\n", " 'PW',\n", " 'PWb',\n", " 'Pa',\n", " 'Padu',\n", " 'Parcmin',\n", " 'Parcsec',\n", " 'Pascal',\n", " 'Pau',\n", " 'Pb',\n", " 'Pbarn',\n", " 'Pbeam',\n", " 'Pbin',\n", " 'Pbit',\n", " 'Pbyte',\n", " 'Pcd',\n", " 'Pchan',\n", " 'Pcount',\n", " 'Pct',\n", " 'Pd',\n", " 'Pdeg',\n", " 'Pdyn',\n", " 'PeV',\n", " 'Perg',\n", " 'Pg',\n", " 'Ph',\n", " 'PiB',\n", " 'Pib',\n", " 'Pibit',\n", " 'Pibyte',\n", " 'Pk',\n", " 'Pl',\n", " 'Plm',\n", " 'Plx',\n", " 'Plyr',\n", " 'Pm',\n", " 'Pmag',\n", " 'Pmin',\n", " 'Pmol',\n", " 'Pohm',\n", " 'Ppc',\n", " 'Pph',\n", " 'Pphoton',\n", " 'Ppix',\n", " 'Ppixel',\n", " 'Prad',\n", " 'PrefixUnit',\n", " 'Ps',\n", " 'Psr',\n", " 'Pu',\n", " 'Pvox',\n", " 'Pvoxel',\n", " 'Pyr',\n", " 'Quantity',\n", " 'QuantityInfo',\n", " 'QuantityInfoBase',\n", " 'R',\n", " 'R_earth',\n", " 'R_jup',\n", " 'R_jupiter',\n", " 'R_sun',\n", " 'Rayleigh',\n", " 'Rearth',\n", " 'Rjup',\n", " 'Rjupiter',\n", " 'Rsun',\n", " 'Ry',\n", " 'S',\n", " 'ST',\n", " 'STflux',\n", " 'STmag',\n", " 'Siemens',\n", " 'SpecificTypeQuantity',\n", " 'St',\n", " 'Sun',\n", " 'T',\n", " 'TA',\n", " 'TAU',\n", " 'TB',\n", " 'TBa',\n", " 'TC',\n", " 'TD',\n", " 'TF',\n", " 'TG',\n", " 'TGal',\n", " 'TH',\n", " 'THz',\n", " 'TJ',\n", " 'TJy',\n", " 'TK',\n", " 'TL',\n", " 'TN',\n", " 'TOhm',\n", " 'TP',\n", " 'TPa',\n", " 'TR',\n", " 'TRy',\n", " 'TS',\n", " 'TSt',\n", " 'TT',\n", " 'TV',\n", " 'TW',\n", " 'TWb',\n", " 'Ta',\n", " 'Tadu',\n", " 'Tarcmin',\n", " 'Tarcsec',\n", " 'Tau',\n", " 'Tb',\n", " 'Tbarn',\n", " 'Tbeam',\n", " 'Tbin',\n", " 'Tbit',\n", " 'Tbyte',\n", " 'Tcd',\n", " 'Tchan',\n", " 'Tcount',\n", " 'Tct',\n", " 'Td',\n", " 'Tdeg',\n", " 'Tdyn',\n", " 'TeV',\n", " 'Terg',\n", " 'Tesla',\n", " 'Tg',\n", " 'Th',\n", " 'TiB',\n", " 'Tib',\n", " 'Tibit',\n", " 'Tibyte',\n", " 'Tk',\n", " 'Tl',\n", " 'Tlm',\n", " 'Tlx',\n", " 'Tlyr',\n", " 'Tm',\n", " 'Tmag',\n", " 'Tmin',\n", " 'Tmol',\n", " 'Tohm',\n", " 'Tpc',\n", " 'Tph',\n", " 'Tphoton',\n", " 'Tpix',\n", " 'Tpixel',\n", " 'Trad',\n", " 'Ts',\n", " 'Tsr',\n", " 'Tu',\n", " 'Tvox',\n", " 'Tvoxel',\n", " 'Tyr',\n", " 'Unit',\n", " 'UnitBase',\n", " 'UnitConversionError',\n", " 'UnitTypeError',\n", " 'UnitsError',\n", " 'UnitsWarning',\n", " 'UnrecognizedUnit',\n", " 'V',\n", " 'Volt',\n", " 'W',\n", " 'Watt',\n", " 'Wb',\n", " 'Weber',\n", " 'YA',\n", " 'YAU',\n", " 'YB',\n", " 'YBa',\n", " 'YC',\n", " 'YD',\n", " 'YF',\n", " 'YG',\n", " 'YGal',\n", " 'YH',\n", " 'YHz',\n", " 'YJ',\n", " 'YJy',\n", " 'YK',\n", " 'YL',\n", " 'YN',\n", " 'YOhm',\n", " 'YP',\n", " 'YPa',\n", " 'YR',\n", " 'YRy',\n", " 'YS',\n", " 'YSt',\n", " 'YT',\n", " 'YV',\n", " 'YW',\n", " 'YWb',\n", " 'Ya',\n", " 'Yadu',\n", " 'Yarcmin',\n", " 'Yarcsec',\n", " 'Yau',\n", " 'Yb',\n", " 'Ybarn',\n", " 'Ybeam',\n", " 'Ybin',\n", " 'Ybit',\n", " 'Ybyte',\n", " 'Ycd',\n", " 'Ychan',\n", " 'Ycount',\n", " 'Yct',\n", " 'Yd',\n", " 'Ydeg',\n", " 'Ydyn',\n", " 'YeV',\n", " 'Yerg',\n", " 'Yg',\n", " 'Yh',\n", " 'Yk',\n", " 'Yl',\n", " 'Ylm',\n", " 'Ylx',\n", " 'Ylyr',\n", " 'Ym',\n", " 'Ymag',\n", " 'Ymin',\n", " 'Ymol',\n", " 'Yohm',\n", " 'Ypc',\n", " 'Yph',\n", " 'Yphoton',\n", " 'Ypix',\n", " 'Ypixel',\n", " 'Yrad',\n", " 'Ys',\n", " 'Ysr',\n", " 'Yu',\n", " 'Yvox',\n", " 'Yvoxel',\n", " 'Yyr',\n", " 'ZA',\n", " 'ZAU',\n", " 'ZB',\n", " 'ZBa',\n", " 'ZC',\n", " 'ZD',\n", " 'ZF',\n", " 'ZG',\n", " 'ZGal',\n", " 'ZH',\n", " 'ZHz',\n", " 'ZJ',\n", " 'ZJy',\n", " 'ZK',\n", " 'ZL',\n", " 'ZN',\n", " 'ZOhm',\n", " 'ZP',\n", " 'ZPa',\n", " 'ZR',\n", " 'ZRy',\n", " 'ZS',\n", " 'ZSt',\n", " 'ZT',\n", " 'ZV',\n", " 'ZW',\n", " 'ZWb',\n", " 'Za',\n", " 'Zadu',\n", " 'Zarcmin',\n", " 'Zarcsec',\n", " 'Zau',\n", " 'Zb',\n", " 'Zbarn',\n", " 'Zbeam',\n", " 'Zbin',\n", " 'Zbit',\n", " 'Zbyte',\n", " 'Zcd',\n", " 'Zchan',\n", " 'Zcount',\n", " 'Zct',\n", " 'Zd',\n", " 'Zdeg',\n", " 'Zdyn',\n", " 'ZeV',\n", " 'Zerg',\n", " 'Zg',\n", " 'Zh',\n", " 'Zk',\n", " 'Zl',\n", " 'Zlm',\n", " 'Zlx',\n", " 'Zlyr',\n", " 'Zm',\n", " 'Zmag',\n", " 'Zmin',\n", " 'Zmol',\n", " 'Zohm',\n", " 'Zpc',\n", " 'Zph',\n", " 'Zphoton',\n", " 'Zpix',\n", " 'Zpixel',\n", " 'Zrad',\n", " 'Zs',\n", " 'Zsr',\n", " 'Zu',\n", " 'Zvox',\n", " 'Zvoxel',\n", " 'Zyr',\n", " '__builtins__',\n", " '__cached__',\n", " '__doc__',\n", " '__file__',\n", " '__loader__',\n", " '__name__',\n", " '__package__',\n", " '__path__',\n", " '__spec__',\n", " 'a',\n", " 'aA',\n", " 'aAU',\n", " 'aB',\n", " 'aBa',\n", " 'aC',\n", " 'aD',\n", " 'aF',\n", " 'aG',\n", " 'aGal',\n", " 'aH',\n", " 'aHz',\n", " 'aJ',\n", " 'aJy',\n", " 'aK',\n", " 'aL',\n", " 'aN',\n", " 'aOhm',\n", " 'aP',\n", " 'aPa',\n", " 'aR',\n", " 'aRy',\n", " 'aS',\n", " 'aSt',\n", " 'aT',\n", " 'aV',\n", " 'aW',\n", " 'aWb',\n", " 'aa',\n", " 'aadu',\n", " 'aarcmin',\n", " 'aarcsec',\n", " 'aau',\n", " 'ab',\n", " 'abA',\n", " 'abC',\n", " 'abampere',\n", " 'abarn',\n", " 'abcoulomb',\n", " 'abeam',\n", " 'abin',\n", " 'abit',\n", " 'abyte',\n", " 'acd',\n", " 'achan',\n", " 'acount',\n", " 'act',\n", " 'ad',\n", " 'add_enabled_equivalencies',\n", " 'add_enabled_units',\n", " 'adeg',\n", " 'adu',\n", " 'adyn',\n", " 'aeV',\n", " 'aerg',\n", " 'ag',\n", " 'ah',\n", " 'ak',\n", " 'al',\n", " 'allclose',\n", " 'alm',\n", " 'alx',\n", " 'alyr',\n", " 'am',\n", " 'amag',\n", " 'amin',\n", " 'amol',\n", " 'amp',\n", " 'ampere',\n", " 'angstrom',\n", " 'annum',\n", " 'aohm',\n", " 'apc',\n", " 'aph',\n", " 'aphoton',\n", " 'apix',\n", " 'apixel',\n", " 'arad',\n", " 'arcmin',\n", " 'arcminute',\n", " 'arcsec',\n", " 'arcsecond',\n", " 'asr',\n", " 'astronomical_unit',\n", " 'astrophys',\n", " 'attoBarye',\n", " 'attoDa',\n", " 'attoDalton',\n", " 'attoDebye',\n", " 'attoFarad',\n", " 'attoGauss',\n", " 'attoHenry',\n", " 'attoHertz',\n", " 'attoJansky',\n", " 'attoJoule',\n", " 'attoKayser',\n", " 'attoKelvin',\n", " 'attoNewton',\n", " 'attoOhm',\n", " 'attoPascal',\n", " 'attoRayleigh',\n", " 'attoSiemens',\n", " 'attoTesla',\n", " 'attoVolt',\n", " 'attoWatt',\n", " 'attoWeber',\n", " 'attoamp',\n", " 'attoampere',\n", " 'attoannum',\n", " 'attoarcminute',\n", " 'attoarcsecond',\n", " 'attoastronomical_unit',\n", " 'attobarn',\n", " 'attobarye',\n", " 'attobit',\n", " 'attobyte',\n", " 'attocandela',\n", " 'attocoulomb',\n", " 'attocount',\n", " 'attoday',\n", " 'attodebye',\n", " 'attodegree',\n", " 'attodyne',\n", " 'attoelectronvolt',\n", " 'attofarad',\n", " 'attogal',\n", " 'attogauss',\n", " 'attogram',\n", " 'attohenry',\n", " 'attohertz',\n", " 'attohour',\n", " 'attohr',\n", " 'attojansky',\n", " 'attojoule',\n", " 'attokayser',\n", " 'attolightyear',\n", " 'attoliter',\n", " 'attolumen',\n", " 'attolux',\n", " 'attometer',\n", " 'attominute',\n", " 'attomole',\n", " 'attonewton',\n", " 'attoparsec',\n", " 'attopascal',\n", " 'attophoton',\n", " 'attopixel',\n", " 'attopoise',\n", " 'attoradian',\n", " 'attorayleigh',\n", " 'attorydberg',\n", " 'attosecond',\n", " 'attosiemens',\n", " 'attosteradian',\n", " 'attostokes',\n", " 'attotesla',\n", " 'attovolt',\n", " 'attovoxel',\n", " 'attowatt',\n", " 'attoweber',\n", " 'attoyear',\n", " 'au',\n", " 'avox',\n", " 'avoxel',\n", " 'ayr',\n", " 'b',\n", " 'bar',\n", " 'barn',\n", " 'barye',\n", " 'beam',\n", " 'beam_angular_area',\n", " 'becquerel',\n", " 'bin',\n", " 'binary_prefixes',\n", " 'bit',\n", " 'bol',\n", " 'brightness_temperature',\n", " 'byte',\n", " 'cA',\n", " 'cAU',\n", " 'cB',\n", " 'cBa',\n", " 'cC',\n", " 'cD',\n", " 'cF',\n", " 'cG',\n", " 'cGal',\n", " 'cH',\n", " 'cHz',\n", " 'cJ',\n", " 'cJy',\n", " 'cK',\n", " 'cL',\n", " 'cN',\n", " 'cOhm',\n", " 'cP',\n", " 'cPa',\n", " 'cR',\n", " 'cRy',\n", " 'cS',\n", " 'cSt',\n", " 'cT',\n", " 'cV',\n", " 'cW',\n", " 'cWb',\n", " 'ca',\n", " 'cadu',\n", " 'candela',\n", " 'carcmin',\n", " 'carcsec',\n", " 'cau',\n", " 'cb',\n", " 'cbarn',\n", " 'cbeam',\n", " 'cbin',\n", " 'cbit',\n", " 'cbyte',\n", " 'ccd',\n", " 'cchan',\n", " 'ccount',\n", " 'cct',\n", " 'cd',\n", " 'cdeg',\n", " 'cdyn',\n", " 'ceV',\n", " 'centiBarye',\n", " 'centiDa',\n", " 'centiDalton',\n", " 'centiDebye',\n", " 'centiFarad',\n", " 'centiGauss',\n", " 'centiHenry',\n", " 'centiHertz',\n", " 'centiJansky',\n", " 'centiJoule',\n", " 'centiKayser',\n", " 'centiKelvin',\n", " 'centiNewton',\n", " 'centiOhm',\n", " 'centiPascal',\n", " 'centiRayleigh',\n", " 'centiSiemens',\n", " 'centiTesla',\n", " 'centiVolt',\n", " 'centiWatt',\n", " 'centiWeber',\n", " 'centiamp',\n", " 'centiampere',\n", " 'centiannum',\n", " 'centiarcminute',\n", " 'centiarcsecond',\n", " 'centiastronomical_unit',\n", " 'centibarn',\n", " 'centibarye',\n", " 'centibit',\n", " 'centibyte',\n", " 'centicandela',\n", " 'centicoulomb',\n", " 'centicount',\n", " 'centiday',\n", " 'centidebye',\n", " 'centidegree',\n", " 'centidyne',\n", " 'centielectronvolt',\n", " 'centifarad',\n", " 'centigal',\n", " 'centigauss',\n", " 'centigram',\n", " 'centihenry',\n", " 'centihertz',\n", " 'centihour',\n", " 'centihr',\n", " 'centijansky',\n", " 'centijoule',\n", " 'centikayser',\n", " 'centilightyear',\n", " 'centiliter',\n", " 'centilumen',\n", " 'centilux',\n", " 'centimeter',\n", " 'centiminute',\n", " 'centimole',\n", " 'centinewton',\n", " 'centiparsec',\n", " 'centipascal',\n", " 'centiphoton',\n", " 'centipixel',\n", " 'centipoise',\n", " 'centiradian',\n", " 'centirayleigh',\n", " 'centirydberg',\n", " 'centisecond',\n", " 'centisiemens',\n", " 'centisteradian',\n", " 'centistokes',\n", " 'centitesla',\n", " 'centivolt',\n", " 'centivoxel',\n", " 'centiwatt',\n", " 'centiweber',\n", " 'centiyear',\n", " 'cerg',\n", " 'cg',\n", " 'cgs',\n", " 'ch',\n", " 'chan',\n", " 'ck',\n", " 'cl',\n", " 'clm',\n", " 'clx',\n", " 'clyr',\n", " 'cm',\n", " 'cmag',\n", " 'cmin',\n", " 'cmol',\n", " 'cohm',\n", " 'core',\n", " 'coulomb',\n", " 'count',\n", " 'cpc',\n", " 'cph',\n", " 'cphoton',\n", " 'cpix',\n", " 'cpixel',\n", " 'crad',\n", " 'cs',\n", " 'csr',\n", " 'ct',\n", " 'cu',\n", " 'curie',\n", " 'cvox',\n", " 'cvoxel',\n", " 'cy',\n", " 'cycle',\n", " 'cyr',\n", " 'd',\n", " 'dA',\n", " 'dAU',\n", " 'dB',\n", " 'dBa',\n", " 'dC',\n", " 'dD',\n", " 'dF',\n", " 'dG',\n", " 'dGal',\n", " 'dH',\n", " 'dHz',\n", " 'dJ',\n", " 'dJy',\n", " 'dK',\n", " 'dL',\n", " 'dN',\n", " 'dOhm',\n", " 'dP',\n", " 'dPa',\n", " 'dR',\n", " 'dRy',\n", " 'dS',\n", " 'dSt',\n", " 'dT',\n", " ...]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dir(u)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To create a quantity, we multiply a value by a unit." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "astropy.units.quantity.Quantity" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "coordinate = 30 * u.deg\n", "type(coordinate)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The result is a `Quantity` object.\n", "\n", "Jupyter knows how to display `Quantities` like this:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/latex": [ "$30 \\; \\mathrm{{}^{\\circ}}$" ], "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "coordinate" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Selecting a rectangle\n", "\n", "Now we'll select a rectangle from -55 to -45 degrees right ascension and -8 to 4 degrees of declination.\n", "\n", "We'll define variables to contain these limits." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "phi1_min = -55\n", "phi1_max = -45\n", "phi2_min = -8\n", "phi2_max = 4" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To represent a rectangle, we'll use two lists of coordinates and multiply by their units." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "phi1_rect = [phi1_min, phi1_min, phi1_max, phi1_max] * u.deg\n", "phi2_rect = [phi2_min, phi2_max, phi2_max, phi2_min] * u.deg" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`phi1_rect` and `phi2_rect` represent the coordinates of the corners of a rectangle. \n", "\n", "But they are in \"[a Heliocentric spherical coordinate system defined by the orbit of the GD1 stream](https://gala-astro.readthedocs.io/en/latest/_modules/gala/coordinates/gd1.html)\"\n", "\n", "In order to use them in a Gaia query, we have to convert them to [International Celestial Reference System](https://en.wikipedia.org/wiki/International_Celestial_Reference_System) (ICRS) coordinates. We can do that by storing the coordinates in a `GD1Koposov10` object provided by [Gala](https://gala-astro.readthedocs.io/en/latest/coordinates/)." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "gala.coordinates.gd1.GD1Koposov10" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import gala.coordinates as gc\n", "\n", "corners = gc.GD1Koposov10(phi1=phi1_rect, phi2=phi2_rect)\n", "type(corners)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can display the result like this:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "corners" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can use `transform_to` to convert to ICRS coordinates." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "astropy.coordinates.builtin_frames.icrs.ICRS" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import astropy.coordinates as coord\n", "\n", "corners_icrs = corners.transform_to(coord.ICRS)\n", "type(corners_icrs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The result is an `ICRS` object." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "corners_icrs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that a rectangle in one coordinate system is not necessarily a rectangle in another. In this example, the result is a polygon." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Selecting a polygon\n", "\n", "In order to use this polygon as part of an ADQL query, we have to convert it to a string with a comma-separated list of coordinates, as in this example:\n", "\n", "```\n", "\"\"\"\n", "POLYGON(143.65, 20.98, \n", " 134.46, 26.39, \n", " 140.58, 34.85, \n", " 150.16, 29.01)\n", "\"\"\"\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`corners_icrs` behaves like a list, so we can use a `for` loop to iterate through the points." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\n", "\n" ] } ], "source": [ "for point in corners_icrs:\n", " print(point)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From that, we can select the coordinates `ra` and `dec`:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "146d16m31.1993s 19d15m42.8754s\n", "135d25m17.902s 25d52m38.594s\n", "141d36m09.5337s 34d18m17.3891s\n", "152d49m00.1576s 27d08m10.0051s\n" ] } ], "source": [ "for point in corners_icrs:\n", " print(point.ra, point.dec)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The results are quantities with units, but if we select the `value` part, we get a dimensionless floating-point number." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "146.27533313607782 19.261909820533692\n", "135.42163944306296 25.87738722767213\n", "141.60264825107333 34.304830296257144\n", "152.81671044675923 27.136112541397996\n" ] } ], "source": [ "for point in corners_icrs:\n", " print(point.ra.value, point.dec.value)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can use string `format` to convert these numbers to strings." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['146.27533313607782, 19.261909820533692',\n", " '135.42163944306296, 25.87738722767213',\n", " '141.60264825107333, 34.304830296257144',\n", " '152.81671044675923, 27.136112541397996']" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "point_base = \"{point.ra.value}, {point.dec.value}\"\n", "\n", "t = [point_base.format(point=point)\n", " for point in corners_icrs]\n", "t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The result is a list of strings, which we can join into a single string using `join`." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'146.27533313607782, 19.261909820533692, 135.42163944306296, 25.87738722767213, 141.60264825107333, 34.304830296257144, 152.81671044675923, 27.136112541397996'" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "point_list = ', '.join(t)\n", "point_list" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that we invoke `join` on a string and pass the list as an argument.\n", "\n", "Before we can assemble the query, we need `columns` again (as we saw in the previous notebook)." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "columns = 'source_id, ra, dec, pmra, pmdec, parallax, parallax_error, radial_velocity'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here's the base for the query, with format specifiers for `columns` and `point_list`." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "query_base = \"\"\"SELECT {columns}\n", "FROM gaiadr2.gaia_source\n", "WHERE parallax < 1\n", " AND bp_rp BETWEEN -0.75 AND 2 \n", " AND 1 = CONTAINS(POINT(ra, dec), \n", " POLYGON({point_list}))\n", "\"\"\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And here's the result:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "SELECT source_id, ra, dec, pmra, pmdec, parallax, parallax_error, radial_velocity\n", "FROM gaiadr2.gaia_source\n", "WHERE parallax < 1\n", " AND bp_rp BETWEEN -0.75 AND 2 \n", " AND 1 = CONTAINS(POINT(ra, dec), \n", " POLYGON(146.27533313607782, 19.261909820533692, 135.42163944306296, 25.87738722767213, 141.60264825107333, 34.304830296257144, 152.81671044675923, 27.136112541397996))\n", "\n" ] } ], "source": [ "query = query_base.format(columns=columns, \n", " point_list=point_list)\n", "print(query)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As always, we should take a minute to proof-read the query before we launch it.\n", "\n", "The result will be bigger than our previous queries, so it will take a little longer." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INFO: Query finished. [astroquery.utils.tap.core]\n", "
\n", " name dtype unit description n_bad \n", "--------------- ------- -------- ------------------------------------------------------------------ ------\n", " source_id int64 Unique source identifier (unique within a particular Data Release) 0\n", " ra float64 deg Right ascension 0\n", " dec float64 deg Declination 0\n", " pmra float64 mas / yr Proper motion in right ascension direction 0\n", " pmdec float64 mas / yr Proper motion in declination direction 0\n", " parallax float64 mas Parallax 0\n", " parallax_error float64 mas Standard error of parallax 0\n", "radial_velocity float64 km / s Radial velocity 139374\n", "Jobid: 1605624329053O\n", "Phase: COMPLETED\n", "Owner: None\n", "Output file: async_20201117094529.vot\n", "Results: None\n" ] } ], "source": [ "job = Gaia.launch_job_async(query)\n", "print(job)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here are the results." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "140340" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results = job.get_results()\n", "len(results)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are more than 100,000 stars in this polygon, but that's a manageable size to work with." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Saving results\n", "\n", "This is the set of stars we'll work with in the next step. But since we have a substantial dataset now, this is a good time to save it.\n", "\n", "Storing the data in a file means we can shut down this notebook and pick up where we left off without running the previous query again.\n", "\n", "Astropy `Table` objects provide `write`, which writes the table to disk." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "filename = 'gd1_results.fits'\n", "results.write(filename, overwrite=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because the filename ends with `fits`, the table is written in the [FITS format](https://en.wikipedia.org/wiki/FITS), which preserves the metadata associated with the table.\n", "\n", "If the file already exists, the `overwrite` argument causes it to be overwritten.\n", "\n", "To see how big the file is, we can use `ls` with the `-lh` option, which prints information about the file including its size in human-readable form." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "-rw-rw-r-- 1 downey downey 8.6M Nov 17 09:45 gd1_results.fits\r\n" ] } ], "source": [ "!ls -lh gd1_results.fits" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The file is about 8.6 MB. If you are using Windows, `ls` might not work; in that case, try:\n", "\n", "```\n", "!dir gd1_results.fits\n", "```" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summary\n", "\n", "In this notebook, we composed more complex queries to select stars within a polygonal region of the sky. Then we downloaded the results and saved them in a FITS file.\n", "\n", "In the next notebook, we'll reload the data from this file and replicate the next step in the analysis, using proper motion to identify stars likely to be in GD-1." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Best practices\n", "\n", "* For measurements with units, use `Quantity` objects that represent units explicitly and check for errors.\n", "\n", "* Use the `format` function to compose queries; it is often faster and less error-prone.\n", "\n", "* Develop queries incrementally: start with something simple, test it, and add a little bit at a time.\n", "\n", "* Once you have a query working, save the data in a local file. If you shut down the notebook and come back to it later, you can reload the file; you don't have to run the query again." ] }, { "cell_type": "raw", "metadata": {}, "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 2 }