mirror of
https://github.com/AllenDowney/AstronomicalData.git
synced 2025-12-25 04:15:38 -08:00
Updating notebooks
This commit is contained in:
Allen Downey
@@ -1,5 +1,45 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "raw",
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"remove-cell"
|
||||
]
|
||||
},
|
||||
"source": [
|
||||
"---\n",
|
||||
"title: \"Coordinate Transformations\"\n",
|
||||
"teaching: 3000\n",
|
||||
"exercises: 0\n",
|
||||
"questions:\n",
|
||||
"\n",
|
||||
"- \"How do we transform celestial coordinates from one frame to another and save results in files?\"\n",
|
||||
"\n",
|
||||
"objectives:\n",
|
||||
"\n",
|
||||
"- \"Use Python string formatting to compose more complex ADQL queries.\"\n",
|
||||
"\n",
|
||||
"- \"Work with coordinates and other quantities that have units.\"\n",
|
||||
"\n",
|
||||
"- \"Download the results of a query and store them in a file.\"\n",
|
||||
"\n",
|
||||
"keypoints:\n",
|
||||
"\n",
|
||||
"- \"For measurements with units, use `Quantity` objects that represent units explicitly and check for errors.\"\n",
|
||||
"\n",
|
||||
"- \"Use the `format` function to compose queries; it is often faster and less error-prone.\"\n",
|
||||
"\n",
|
||||
"- \"Develop queries incrementally: start with something simple, test it, and add a little bit at a time.\"\n",
|
||||
"\n",
|
||||
"- \"Once you have a query working, save the data in a local file. If you shut down the notebook and come back to it later, you can reload the file; you don't have to run the query again.\"\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"FIXME\n",
|
||||
"\n",
|
||||
"{% include links.md %}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -1955,6 +1995,7 @@
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"celltoolbar": "Tags",
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
|
||||
@@ -1,5 +1,49 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "raw",
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"remove-cell"
|
||||
]
|
||||
},
|
||||
"source": [
|
||||
"---\n",
|
||||
"title: \"Plotting and Pandas\"\n",
|
||||
"teaching: 3000\n",
|
||||
"exercises: 0\n",
|
||||
"questions:\n",
|
||||
"\n",
|
||||
"- \"How do we make scatter plots in Matplotlib?\"\n",
|
||||
"\n",
|
||||
"- \"How do we store data in a Pandas `DataFrame`?\"\n",
|
||||
"\n",
|
||||
"objectives:\n",
|
||||
"\n",
|
||||
"- \"Select rows and columns from an Astropy `Table`.\"\n",
|
||||
"\n",
|
||||
"- \"Use Matplotlib to make a scatter plot.\"\n",
|
||||
"\n",
|
||||
"- \"Use Gala to transform coordinates.\"\n",
|
||||
"\n",
|
||||
"- \"Make a Pandas `DataFrame` and use a Boolean `Series` to select rows.\"\n",
|
||||
"\n",
|
||||
"- \"Save a `DataFrame` in an HDF5 file.\"\n",
|
||||
"\n",
|
||||
"keypoints:\n",
|
||||
"\n",
|
||||
"- \"When you make a scatter plot, adjust the size of the markers and their transparency so the figure is not overplotted; otherwise it can misrepresent the data badly.\n",
|
||||
"\n",
|
||||
"- \"For simple scatter plots in Matplotlib, `plot` is faster than `scatter`.\n",
|
||||
"\n",
|
||||
"- \"An Astropy `Table` and a Pandas `DataFrame` are similar in many ways and they provide many of the same functions. They have pros and cons, but for many projects, either one would be a reasonable choice.\"\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"FIXME\n",
|
||||
"\n",
|
||||
"{% include links.md %}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -1075,7 +1119,7 @@
|
||||
"source": [
|
||||
"## Exploring data\n",
|
||||
"\n",
|
||||
"One benefit of using Pandas is that it provides function for exploring the data and checking for problems.\n",
|
||||
"One benefit of using Pandas is that it provides functions for exploring the data and checking for problems.\n",
|
||||
"\n",
|
||||
"One of the most useful of these functions is `describe`, which computes summary statistics for each column."
|
||||
]
|
||||
|
||||
559
04_select.ipynb
559
04_select.ipynb
File diff suppressed because one or more lines are too long
400
05_join.ipynb
400
05_join.ipynb
@@ -1,5 +1,35 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "raw",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"title: \"Join\"\n",
|
||||
"teaching: 3000\n",
|
||||
"exercises: 0\n",
|
||||
"questions:\n",
|
||||
"\n",
|
||||
"- \"How do we use `JOIN` to combine information from multiple tables?\"\n",
|
||||
"\n",
|
||||
"objectives:\n",
|
||||
"\n",
|
||||
"- \"Upload a table to the Gaia server.\"\n",
|
||||
"\n",
|
||||
"- \"Write ADQL queries involving `JOIN` operations.\"\n",
|
||||
"\n",
|
||||
"keypoints:\n",
|
||||
"\n",
|
||||
"- \"Use `JOIN` operations to combine data from multiple tables in a databased, using some kind of identifier to match up records from one table with records from another.\"\n",
|
||||
"\n",
|
||||
"* \"This is another example of a practice we saw in the previous notebook, moving the computation to the data.\"\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"FIXME\n",
|
||||
"\n",
|
||||
"{% include links.md %}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -10,11 +40,11 @@
|
||||
"\n",
|
||||
"As a continuing example, we will replicate part of the analysis in a recent paper, \"[Off the beaten path: Gaia reveals GD-1 stars outside of the main stream](https://arxiv.org/abs/1805.00425)\" by Adrian M. Price-Whelan and Ana Bonaca.\n",
|
||||
"\n",
|
||||
"Picking up where we left off, the next step in the analysis is to select candidate stars based on photometry. The following figure from the paper is a color-magnitude diagram for the stars selected based on proper motion:\n",
|
||||
"Picking up where we left off, the next step in the analysis is to select candidate stars based on photometry data. The following figure from the paper is a color-magnitude diagram for the stars selected based on proper motion:\n",
|
||||
"\n",
|
||||
"<img width=\"300\" src=\"https://github.com/datacarpentry/astronomy-python/raw/gh-pages/fig/gd1-3.png\">\n",
|
||||
"\n",
|
||||
"In red is a theoretical isochrone, showing where we expect the stars in GD-1 to fall based on the metallicity and age of their original globular cluster. \n",
|
||||
"In red is a [stellar isochrone](https://en.wikipedia.org/wiki/Stellar_isochrone), showing where we expect the stars in GD-1 to fall based on the metallicity and age of their original globular cluster. \n",
|
||||
"\n",
|
||||
"By selecting stars in the shaded area, we can further distinguish the main sequence of GD-1 from younger background stars."
|
||||
]
|
||||
@@ -311,7 +341,7 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n",
|
||||
"<!-- Produced with astropy.io.votable version 4.0.2\r\n",
|
||||
"<!-- Produced with astropy.io.votable version 4.2\r\n",
|
||||
" http://www.astropy.org/ -->\r\n",
|
||||
"<VOTABLE version=\"1.4\" xmlns=\"http://www.ivoa.net/xml/VOTable/v1.4\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:noNamespaceSchemaLocation=\"http://www.ivoa.net/xml/VOTable/v1.4\">\r\n",
|
||||
" <RESOURCE type=\"results\">\r\n",
|
||||
@@ -350,7 +380,7 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"-rw-rw-r-- 1 downey downey 396K Nov 18 19:21 candidate_df.xml\r\n"
|
||||
"-rw-rw-r-- 1 downey downey 396K Dec 10 11:33 candidate_df.xml\r\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@@ -373,7 +403,9 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Exercise:** There's a gotcha here we want to warn you about. Why do you think we used double brackets to specify the column we wanted? What happens if you use single brackets?\n",
|
||||
"### Exercise\n",
|
||||
"\n",
|
||||
"There's a gotcha here we want to warn you about. Why do you think we used double brackets to specify the column we wanted? What happens if you use single brackets?\n",
|
||||
"\n",
|
||||
"Run these cells to find out."
|
||||
]
|
||||
@@ -426,6 +458,8 @@
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# This line is commented out because it would cause an error\n",
|
||||
"\n",
|
||||
"# writeto(column, 'candidate_df.xml')"
|
||||
]
|
||||
},
|
||||
@@ -517,7 +551,7 @@
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<i>Table length=7346</i>\n",
|
||||
"<table id=\"table140178070186304\" class=\"table-striped table-bordered table-condensed\">\n",
|
||||
"<table id=\"table140668290919488\" class=\"table-striped table-bordered table-condensed\">\n",
|
||||
"<thead><tr><th>source_id</th></tr></thead>\n",
|
||||
"<thead><tr><th>int64</th></tr></thead>\n",
|
||||
"<tr><td>635559124339440000</td></tr>\n",
|
||||
@@ -655,7 +689,7 @@
|
||||
"query1 = \"\"\"SELECT *\n",
|
||||
"FROM gaiadr2.panstarrs1_best_neighbour as best\n",
|
||||
"JOIN tap_upload.candidate_df as candidate_df\n",
|
||||
"ON best.source_id = candidate_df.source_id\n",
|
||||
" ON best.source_id = candidate_df.source_id\n",
|
||||
"\"\"\""
|
||||
]
|
||||
},
|
||||
@@ -713,7 +747,7 @@
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<i>Table length=3724</i>\n",
|
||||
"<table id=\"table140178046436112\" class=\"table-striped table-bordered table-condensed\">\n",
|
||||
"<table id=\"table140668268803072\" class=\"table-striped table-bordered table-condensed\">\n",
|
||||
"<thead><tr><th>source_id</th><th>original_ext_source_id</th><th>angular_distance</th><th>number_of_neighbours</th><th>number_of_mates</th><th>best_neighbour_multiplicity</th><th>gaia_astrometric_params</th><th>source_id_2</th></tr></thead>\n",
|
||||
"<thead><tr><th></th><th></th><th>arcsec</th><th></th><th></th><th></th><th></th><th></th></tr></thead>\n",
|
||||
"<thead><tr><th>int64</th><th>int64</th><th>float64</th><th>int32</th><th>int16</th><th>int16</th><th>int16</th><th>int64</th></tr></thead>\n",
|
||||
@@ -819,7 +853,7 @@
|
||||
"source": [
|
||||
"Because one of the column names appears in both tables, the second instance of `source_id` has been appended with the suffix `_2`.\n",
|
||||
"\n",
|
||||
"The length of the results table is about 2000, which means we were not able to find matches for all stars in the list of candidate_df."
|
||||
"The length of `results1` is about 3000, which means we were not able to find matches for all stars in the list of candidates."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -846,9 +880,7 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To get more information about the matching process, we can inspect `best_neighbour_multiplicity`, which indicates for each star in Gaia how many stars in Pan-STARRS are equally likely matches.\n",
|
||||
"\n",
|
||||
"For this kind of data exploration, we'll convert a column from the table to a Pandas `Series` so we can use `value_counts`, which counts the number of times each value appears in a `Series`, like a histogram."
|
||||
"To get more information about the matching process, we can inspect `best_neighbour_multiplicity`, which indicates for each star in Gaia how many stars in Pan-STARRS are equally likely matches."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -858,9 +890,63 @@
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<MaskedColumn name='best_neighbour_multiplicity' dtype='int16' description='Number of neighbours with same probability as best neighbour' length=3724>\n",
|
||||
"<table>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>...</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"<tr><td>1</td></tr>\n",
|
||||
"</table>"
|
||||
],
|
||||
"text/plain": [
|
||||
"1 3724\n",
|
||||
"dtype: int64"
|
||||
"<MaskedColumn name='best_neighbour_multiplicity' dtype='int16' description='Number of neighbours with same probability as best neighbour' length=3724>\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
"...\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1\n",
|
||||
" 1"
|
||||
]
|
||||
},
|
||||
"execution_count": 22,
|
||||
@@ -869,21 +955,16 @@
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"nn = pd.Series(results1['best_neighbour_multiplicity'])\n",
|
||||
"nn.value_counts()"
|
||||
"results1['best_neighbour_multiplicity']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The result shows that `1` is the only value in the `Series`, appearing xxx times.\n",
|
||||
"It looks like most of the values are `1`, which is good; that means that for each candidate star we have identified exactly one source in Pan-STARRS that is likely to be the same star.\n",
|
||||
"\n",
|
||||
"That means that in every case where a match was found, the matching algorithm identified a single neighbor as the most likely match.\n",
|
||||
"\n",
|
||||
"Similarly, `number_of_mates` indicates the number of other stars in Gaia that match with the same star in Pan-STARRS."
|
||||
"To check whether there are any values other than `1`, we can convert this column to a Pandas `Series` and use `describe`, which we saw in in Lesson 3."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -894,8 +975,15 @@
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"0 3724\n",
|
||||
"dtype: int64"
|
||||
"count 3724.0\n",
|
||||
"mean 1.0\n",
|
||||
"std 0.0\n",
|
||||
"min 1.0\n",
|
||||
"25% 1.0\n",
|
||||
"50% 1.0\n",
|
||||
"75% 1.0\n",
|
||||
"max 1.0\n",
|
||||
"dtype: float64"
|
||||
]
|
||||
},
|
||||
"execution_count": 23,
|
||||
@@ -904,17 +992,57 @@
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"nm = pd.Series(results1['number_of_mates'])\n",
|
||||
"nm.value_counts()"
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"multiplicity = pd.Series(results1['best_neighbour_multiplicity'])\n",
|
||||
"multiplicity.describe()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"For this set of candidate_df, almost all of the stars we've selected from Pan-STARRS are only matched with a single star in the Gaia catalog.\n",
|
||||
"In fact, `1` is the only value in the `Series`, so every candidate star has a single best match.\n",
|
||||
"\n",
|
||||
"**Detail** The table also contains `number_of_neighbors` which is the number of stars in Pan-STARRS that match in terms of position, before using other critieria to choose the most likely match."
|
||||
"Similarly, `number_of_mates` indicates the number of *other* stars in Gaia that match with the same star in Pan-STARRS."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 24,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"count 3724.0\n",
|
||||
"mean 0.0\n",
|
||||
"std 0.0\n",
|
||||
"min 0.0\n",
|
||||
"25% 0.0\n",
|
||||
"50% 0.0\n",
|
||||
"75% 0.0\n",
|
||||
"max 0.0\n",
|
||||
"dtype: float64"
|
||||
]
|
||||
},
|
||||
"execution_count": 24,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"mates = pd.Series(results1['number_of_mates'])\n",
|
||||
"mates.describe()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"All values in this column are `0`, which means that for each match we found in Pan-STARRS, there are no other stars in Gaia that also match. \n",
|
||||
"\n",
|
||||
"**Detail:** The table also contains `number_of_neighbors` which is the number of stars in Pan-STARRS that match in terms of position, before using other criteria to choose the most likely match."
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -935,14 +1063,21 @@
|
||||
"\n",
|
||||
"4. Run the query using the uploaded table.\n",
|
||||
"\n",
|
||||
"Since we've done everything here before, we'll do these steps as an exercise.\n",
|
||||
"Since we've done everything here before, we'll do these steps as an exercise."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Exercise\n",
|
||||
"\n",
|
||||
"**Exercise:** Select `source_id` and `original_ext_source_id` from `results1` and write the resulting table as a file named `external.xml`."
|
||||
"Select `source_id` and `original_ext_source_id` from `results1` and write the resulting table as a file named `external.xml`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 24,
|
||||
"execution_count": 25,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"hide-cell"
|
||||
@@ -965,7 +1100,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 25,
|
||||
"execution_count": 26,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -973,7 +1108,7 @@
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n",
|
||||
"<!-- Produced with astropy.io.votable version 4.0.2\r\n",
|
||||
"<!-- Produced with astropy.io.votable version 4.2\r\n",
|
||||
" http://www.astropy.org/ -->\r\n",
|
||||
"<VOTABLE version=\"1.4\" xmlns=\"http://www.ivoa.net/xml/VOTable/v1.4\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:noNamespaceSchemaLocation=\"http://www.ivoa.net/xml/VOTable/v1.4\">\r\n",
|
||||
" <RESOURCE type=\"results\">\r\n",
|
||||
@@ -993,7 +1128,9 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Exercise:** Read [the documentation of the Pan-STARRS table](https://gea.esac.esa.int/archive/documentation/GDR2/Gaia_archive/chap_datamodel/sec_dm_external_catalogues/ssec_dm_panstarrs1_original_valid.html) and make note of `obj_id`, which contains the object IDs we'll use to find the rows we want.\n",
|
||||
"### Exercise\n",
|
||||
"\n",
|
||||
"Read [the documentation of the Pan-STARRS table](https://gea.esac.esa.int/archive/documentation/GDR2/Gaia_archive/chap_datamodel/sec_dm_external_catalogues/ssec_dm_panstarrs1_original_valid.html) and make note of `obj_id`, which contains the object IDs we'll use to find the rows we want.\n",
|
||||
"\n",
|
||||
"Write a query that uses each value of `original_ext_source_id` from the uploaded table to find a row in `gaiadr2.panstarrs1_original_valid` with the same value in `obj_id`, and select all columns from both tables.\n",
|
||||
"\n",
|
||||
@@ -1017,23 +1154,6 @@
|
||||
"Hint: When you select a column from a join, you have to specify which table the column is in."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 26,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"hide-cell"
|
||||
]
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Solution\n",
|
||||
"\n",
|
||||
"query2 = \"\"\"SELECT *\n",
|
||||
"FROM tap_upload.external as external\n",
|
||||
"\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 27,
|
||||
@@ -1046,75 +1166,46 @@
|
||||
"source": [
|
||||
"# Solution\n",
|
||||
"\n",
|
||||
"# First test\n",
|
||||
"\n",
|
||||
"query2 = \"\"\"SELECT *\n",
|
||||
"FROM tap_upload.external as external\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"# Second test\n",
|
||||
"\n",
|
||||
"query2 = \"\"\"SELECT TOP 10 *\n",
|
||||
"FROM gaiadr2.panstarrs1_original_valid\n",
|
||||
"\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 28,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"hide-cell"
|
||||
]
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Solution\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"# Third test\n",
|
||||
"\n",
|
||||
"query2 = \"\"\"SELECT *\n",
|
||||
"FROM gaiadr2.panstarrs1_original_valid as ps\n",
|
||||
"JOIN tap_upload.external as external\n",
|
||||
"ON ps.obj_id = external.original_ext_source_id\n",
|
||||
"\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 29,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"hide-cell"
|
||||
]
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Solution\n",
|
||||
" ON ps.obj_id = external.original_ext_source_id\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"# Complete query\n",
|
||||
"\n",
|
||||
"query2 = \"\"\"SELECT\n",
|
||||
"external.source_id, ps.g_mean_psf_mag, ps.i_mean_psf_mag\n",
|
||||
"FROM gaiadr2.panstarrs1_original_valid as ps\n",
|
||||
"JOIN tap_upload.external as external\n",
|
||||
"ON ps.obj_id = external.original_ext_source_id\n",
|
||||
" ON ps.obj_id = external.original_ext_source_id\n",
|
||||
"\"\"\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 30,
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"SELECT\n",
|
||||
"external.source_id, ps.g_mean_psf_mag, ps.i_mean_psf_mag\n",
|
||||
"FROM gaiadr2.panstarrs1_original_valid as ps\n",
|
||||
"JOIN tap_upload.external as external\n",
|
||||
"ON ps.obj_id = external.original_ext_source_id\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(query2)"
|
||||
"Here's how we launch the job and get the results."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 31,
|
||||
"execution_count": 28,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
@@ -1133,14 +1224,14 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 32,
|
||||
"execution_count": 29,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<i>Table length=3724</i>\n",
|
||||
"<table id=\"table140178044101488\" class=\"table-striped table-bordered table-condensed\">\n",
|
||||
"<table id=\"table140668299958448\" class=\"table-striped table-bordered table-condensed\">\n",
|
||||
"<thead><tr><th>source_id</th><th>g_mean_psf_mag</th><th>i_mean_psf_mag</th></tr></thead>\n",
|
||||
"<thead><tr><th></th><th></th><th>mag</th></tr></thead>\n",
|
||||
"<thead><tr><th>int64</th><th>float64</th><th>float64</th></tr></thead>\n",
|
||||
@@ -1194,7 +1285,7 @@
|
||||
"612256418500423168 20.8715991973877 19.9612007141113"
|
||||
]
|
||||
},
|
||||
"execution_count": 32,
|
||||
"execution_count": 29,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
@@ -1208,13 +1299,108 @@
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**Challenge exercise**\n",
|
||||
"### Exercise\n",
|
||||
"\n",
|
||||
"Do both joins in one query.\n",
|
||||
"Optional Challenge: Do both joins in one query.\n",
|
||||
"\n",
|
||||
"There's an [example here](https://github.com/smoh/Getting-started-with-Gaia/blob/master/gaia-adql-snippets.md) you could start with."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 30,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"INFO: Query finished. [astroquery.utils.tap.core]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<i>Table length=3724</i>\n",
|
||||
"<table id=\"table140668275467456\" class=\"table-striped table-bordered table-condensed\">\n",
|
||||
"<thead><tr><th>source_id</th><th>g_mean_psf_mag</th><th>i_mean_psf_mag</th></tr></thead>\n",
|
||||
"<thead><tr><th></th><th></th><th>mag</th></tr></thead>\n",
|
||||
"<thead><tr><th>int64</th><th>float64</th><th>float64</th></tr></thead>\n",
|
||||
"<tr><td>635860218726658176</td><td>17.8978004455566</td><td>17.5174007415771</td></tr>\n",
|
||||
"<tr><td>635674126383965568</td><td>19.2873001098633</td><td>17.6781005859375</td></tr>\n",
|
||||
"<tr><td>635535454774983040</td><td>16.9237995147705</td><td>16.478099822998</td></tr>\n",
|
||||
"<tr><td>635497276810313600</td><td>19.9242000579834</td><td>18.3339996337891</td></tr>\n",
|
||||
"<tr><td>635614168640132864</td><td>16.1515998840332</td><td>14.6662998199463</td></tr>\n",
|
||||
"<tr><td>635598607974369792</td><td>16.5223999023438</td><td>16.1375007629395</td></tr>\n",
|
||||
"<tr><td>635737661835496576</td><td>14.5032997131348</td><td>13.9849004745483</td></tr>\n",
|
||||
"<tr><td>635850945892748672</td><td>16.5174999237061</td><td>16.0450000762939</td></tr>\n",
|
||||
"<tr><td>635600532119713664</td><td>20.4505996704102</td><td>19.5177001953125</td></tr>\n",
|
||||
"<tr><td>...</td><td>...</td><td>...</td></tr>\n",
|
||||
"<tr><td>612241781249124608</td><td>20.2343997955322</td><td>18.6518001556396</td></tr>\n",
|
||||
"<tr><td>612332147361443072</td><td>21.3848991394043</td><td>20.3076000213623</td></tr>\n",
|
||||
"<tr><td>612426744016802432</td><td>17.8281002044678</td><td>17.4281005859375</td></tr>\n",
|
||||
"<tr><td>612331739340341760</td><td>21.8656997680664</td><td>19.5223007202148</td></tr>\n",
|
||||
"<tr><td>612282738058264960</td><td>22.5151996612549</td><td>19.9743995666504</td></tr>\n",
|
||||
"<tr><td>612386332668697600</td><td>19.3792991638184</td><td>17.9923000335693</td></tr>\n",
|
||||
"<tr><td>612296172717818624</td><td>17.4944000244141</td><td>16.926700592041</td></tr>\n",
|
||||
"<tr><td>612250375480101760</td><td>15.3330001831055</td><td>14.6280002593994</td></tr>\n",
|
||||
"<tr><td>612394926899159168</td><td>16.4414005279541</td><td>15.8212003707886</td></tr>\n",
|
||||
"<tr><td>612256418500423168</td><td>20.8715991973877</td><td>19.9612007141113</td></tr>\n",
|
||||
"</table>"
|
||||
],
|
||||
"text/plain": [
|
||||
"<Table length=3724>\n",
|
||||
" source_id g_mean_psf_mag i_mean_psf_mag \n",
|
||||
" mag \n",
|
||||
" int64 float64 float64 \n",
|
||||
"------------------ ---------------- ----------------\n",
|
||||
"635860218726658176 17.8978004455566 17.5174007415771\n",
|
||||
"635674126383965568 19.2873001098633 17.6781005859375\n",
|
||||
"635535454774983040 16.9237995147705 16.478099822998\n",
|
||||
"635497276810313600 19.9242000579834 18.3339996337891\n",
|
||||
"635614168640132864 16.1515998840332 14.6662998199463\n",
|
||||
"635598607974369792 16.5223999023438 16.1375007629395\n",
|
||||
"635737661835496576 14.5032997131348 13.9849004745483\n",
|
||||
"635850945892748672 16.5174999237061 16.0450000762939\n",
|
||||
"635600532119713664 20.4505996704102 19.5177001953125\n",
|
||||
" ... ... ...\n",
|
||||
"612241781249124608 20.2343997955322 18.6518001556396\n",
|
||||
"612332147361443072 21.3848991394043 20.3076000213623\n",
|
||||
"612426744016802432 17.8281002044678 17.4281005859375\n",
|
||||
"612331739340341760 21.8656997680664 19.5223007202148\n",
|
||||
"612282738058264960 22.5151996612549 19.9743995666504\n",
|
||||
"612386332668697600 19.3792991638184 17.9923000335693\n",
|
||||
"612296172717818624 17.4944000244141 16.926700592041\n",
|
||||
"612250375480101760 15.3330001831055 14.6280002593994\n",
|
||||
"612394926899159168 16.4414005279541 15.8212003707886\n",
|
||||
"612256418500423168 20.8715991973877 19.9612007141113"
|
||||
]
|
||||
},
|
||||
"execution_count": 30,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Solution\n",
|
||||
"\n",
|
||||
"query3 = \"\"\"SELECT\n",
|
||||
"candidate_df.source_id, ps.g_mean_psf_mag, ps.i_mean_psf_mag\n",
|
||||
"FROM tap_upload.candidate_df as candidate_df\n",
|
||||
"JOIN gaiadr2.panstarrs1_best_neighbour as best\n",
|
||||
" ON best.source_id = candidate_df.source_id\n",
|
||||
"JOIN gaiadr2.panstarrs1_original_valid as ps\n",
|
||||
" ON ps.obj_id = best.original_ext_source_id\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"job3 = Gaia.launch_job_async(query=query3, \n",
|
||||
" upload_resource='candidate_df.xml', \n",
|
||||
" upload_table_name='candidate_df')\n",
|
||||
"\n",
|
||||
"results3 = job3.get_results()\n",
|
||||
"results3"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
@@ -1226,7 +1412,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 33,
|
||||
"execution_count": 31,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
@@ -1243,14 +1429,14 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 34,
|
||||
"execution_count": 32,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"-rw-rw-r-- 1 downey downey 96K Nov 18 19:22 gd1_photo.fits\r\n"
|
||||
"-rw-rw-r-- 1 downey downey 96K Dec 10 11:34 gd1_photo.fits\r\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
||||
1695
06_photo.ipynb
1695
06_photo.ipynb
File diff suppressed because one or more lines are too long
@@ -1,5 +1,31 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "raw",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"---\n",
|
||||
"title: \"Title\"\n",
|
||||
"teaching: 3000\n",
|
||||
"exercises: 0\n",
|
||||
"questions:\n",
|
||||
"\n",
|
||||
"- \"Question?\"\n",
|
||||
"\n",
|
||||
"objectives:\n",
|
||||
"\n",
|
||||
"- \"Objective.\"\n",
|
||||
"\n",
|
||||
"keypoints:\n",
|
||||
"\n",
|
||||
"- \"Keypoint.\"\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"FIXME\n",
|
||||
"\n",
|
||||
"{% include links.md %}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
|
||||
Reference in New Issue
Block a user