Query and Access Data in a Gen3 Data Commons¶


by Chris Meyer, PhD

Manager of Data and User Services at the Center for Translational Data Science at University of Chicago

June 2022

Introduction¶


  • This notebook is intended to demonstrate a variety of ways to access file objects and structured data (aka "metadata") in a Gen3 data commons.
  • File objects are accessed via their "data GUID" aka "object_id", which is a unique identifier that is associated with a storage_url in the file index (https://data.midrc.org/index/index). Users must be authorized to access a file in order to download it via the object_id.
  • Structured data in a Gen3 Data Commons is imported into Postgres via the "sheepdog" service and must conform to the data model. The data model is a relational model that consists of tables or "nodes" that are related to one another via foreign keys so that the model can be thought of as a graph of nodes that are linked to each other. Each node in the model contains certain properties (keys) that store data of a particular type (values).
  • The "sheepdog" service can export tables of data from a particular node of a data project. This is the simplest way to access "all" the data in a Gen3 data commons.
  • Queries can be constructed to target specific types of data in Postgres and are handled by the "peregrine" graphQL service.
  • Structured data can also be transformed via an "ETL" (extract, transform, load) process that takes the complex relationships between nodes and "flattens" the data into a single table, which is stored in an ElasticSearch (ES) database that can be queried using the "guppy" graphQL service. These ES tables are what the data exploration app of the Gen3 data-portal is based on.
In [3]:
## May need to install these Python packages. 
## We should add them to the base workspace image so this is unnecessary..

!pip install --upgrade --ignore-installed PyYAML
!pip install --upgrade pip
!pip install --upgrade gen3
!pip install pydicom
In [6]:
# Import Python Packages and scripts
import pandas as pd
import sys, os
import gen3
import pydicom
from io import StringIO


from gen3.submission import Gen3Submission
from gen3.auth import Gen3Auth
from gen3.index import Gen3Index
from gen3.query import Gen3Query
In [30]:
# Import some custom Python scripts from personal GitHub repo
home_dir = "/home/jovyan/pd"
gen3_dir = "{}/gen3".format(home_dir)
os.system('mkdir -p {}'.format(gen3_dir))
os.chdir(gen3_dir)

os.system("wget https://raw.githubusercontent.com/cgmeyer/gen3sdk-python/master/expansion/expansion.py -O /home/jovyan/pd/gen3/expansion.py")
%run /home/jovyan/pd/gen3/expansion.py
In [31]:
## Initiate instances of the Gen3 SDK Classes using the WTS (no need for uploading credentials.json)

api = 'https://data.midrc.org'
auth = Gen3Auth(api) # authentication class; 
## if using outside of a Gen3 workspace, you must initialize by passing your credentials.json file downloaded from your Profile page. See below:
# cred = '/home/jovyan/gen3/midrc-credentials.json'
# auth = Gen3Auth(api, refresh_file=cred)
sub = Gen3Submission(api, auth) # submission class, sheepdog and peregrine (postgres) functions
query = Gen3Query(auth) # query class, guppy (elasticsearch) functions
exp = Gen3Expansion(api,auth,sub) # class with some custom scripts that I wrote
exp.get_project_ids()
[2022-06-22 21:14:57,584][WARNING] Unable to find refresh_file
Getting all project_ids you have access to in the data commons.
['Open-A1', 'Open-A1_PETAL_REDCORAL', 'Open-R1']
Out[31]:
['Open-A1', 'Open-A1_PETAL_REDCORAL', 'Open-R1']
In [4]:
# # for local runs:
# %run /Users/christopher/Documents/GitHub/cgmeyer/gen3sdk-python/expansion/expansion.py
# api = 'https://data.midrc.org'
# cred = '/Users/christopher/Downloads/midrc-credentials.json'
# auth = Gen3Auth(api,refresh_file=cred)
# sub = Gen3Submission(api, auth)
# query = Gen3Query(auth)
# exp = Gen3Expansion(api,auth,sub)
# exp.get_project_ids()

Accessing structured data in Postgres using sheepdog exports¶


  • Probably the most straight-forward way to access structured data in a Gen3 Data Commons is to simply export the table of data using the sheepdog service (https://petstore.swagger.io/?url=https://raw.githubusercontent.com/uc-cdis/sheepdog/master/openapi/swagger.yml#/export/post__program___project__export).
  • The Gen3SDK has a function Gen3Submission.export_node() for exporting entire tables of data from Postgres: https://github.com/uc-cdis/gen3sdk-python/blob/8196cf4b76a65d0b9b31c8637a18dfac2a911b56/gen3/submission.py#L361
    • This function will export all records in a particular node of a specified project, and one can then use standard Python / R (etc.) tools to do the filtering and cohort building.
  • Note: This export function is also accesible in the data-portal by navigating to a data project's URL, e.g., https://data.midrc.org/Open-A1, clicking a node in the graph, and then clicking the "Download All" button.
    • For example: https://data.midrc.org/Open-A1/search?node_type=measurement
    • Or, you can enter this URL in your browser, for example: https://data.midrc.org/api/v0/submission//Open/A1/export?node_label=measurement&format=tsv
In [9]:
## Example of exporting a table of data using the `Gen3Submission.export_node()` function
cases = sub.export_node(program='Open',project='A1',node_type='case',fileformat='tsv')
df = pd.read_csv(StringIO(cases), sep='\t', header=0)
display(df)
type id project_id submitter_id age_at_index age_at_index_gt89 case_ids country_of_origin covid19_positive data_submission_guid ... icu_indicator index_event race sex site_id treatment_info ventilator_indicator zip datasets.id datasets.submitter_id
0 case 00019c92-8598-4d94-84d2-3abf2b132dc9 Open-A1 10000364-1164071 27.0 NaN NaN NaN Yes NaN ... NaN First COVID test Black or African American Female 10000364.0 NaN NaN 272.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
1 case 000465e0-4c4f-48e4-8974-95c045e333cb Open-A1 10000364-5222709 55.0 NaN NaN NaN No NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 272.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
2 case 0010129a-8aa1-4916-834a-c4ebcdaf44d5 Open-A1 10003752-Zlh3OsFFjEC7bnFqpHZgIw 76.0 NaN 10003752-Zlh3OsFFjEC7bnFqpHZgIw NaN Yes NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
3 case 001c584d-b7a1-42d0-a204-c3336683d4e9 Open-A1 10000364-1647020 56.0 NaN 10000364-1647020 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 274.0 14b6b707-15a8-4379-bdca-b7d9eea1b35e,f7bb0d11-... ACR_20210823,ACR_20211020
4 case 00373c30-18f5-4afa-9a96-525bec5e76f0 Open-A1 10000364-5837269 21.0 NaN 10000364-5837269 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 272.0 f670ecfb-bc34-4c2d-b15d-4c2762268904 ACR_20210827
5 case 003ef620-52c8-4cdc-a7a9-c8106edc51bd Open-A1 10000364-1516198 48.0 NaN 10000364-1516198 United States of America Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 0.0 9bf8adc9-008a-4863-9e8e-bd051555eea6 ACR_20210915
6 case 00456adc-9ed7-44f9-b6d1-50284656e20e Open-A1 10000364-655301 71.0 NaN 10000364-655301 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 272.0 f7bb0d11-a6e8-4504-bc3b-e57bf164e633 ACR_20211020
7 case 00458e57-b582-4bdd-9134-bcabe90c6d2e Open-A1 10000364-2611644 83.0 NaN 10000364-2611644 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 0.0 f7bb0d11-a6e8-4504-bc3b-e57bf164e633,2d761222-... ACR_20211020,ACR_20211115
8 case 004b0d6f-8ff5-4549-b0fc-a14354b737eb Open-A1 10000364-948394 39.0 NaN 10000364-948394 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 271.0 14b6b707-15a8-4379-bdca-b7d9eea1b35e ACR_20210823
9 case 00516d40-dcae-4f33-801c-34838e96d24c Open-A1 10003752-ihwNEEhSFkmIx0dTcN3w 72.0 NaN NaN NaN Yes NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
10 case 0058eb1d-aaa2-4d64-8db5-2e292852a661 Open-A1 10000364-5968043 77.0 NaN 10000364-5968043 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 0.0 2d761222-3669-4a77-9473-342c369fcc40,f7bb0d11-... ACR_20211115,ACR_20211020
11 case 005e3d9a-0a15-4f86-b5e3-e8502c95bde9 Open-A1 10000364-2379811 65.0 NaN 10000364-2379811 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 0.0 2d761222-3669-4a77-9473-342c369fcc40,f7bb0d11-... ACR_20211115,ACR_20211020
12 case 00731046-e22f-46e0-b69e-961d911fe1ec Open-A1 10003752-BDZwCSpvl0KgMPTRYWHAwg 68.0 NaN NaN NaN Yes NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
13 case 0077951b-38f9-48f6-9791-4fc897b2ca14 Open-A1 10003752-ynnax8heZ0cbuEqZMOVg 72.0 NaN NaN NaN No NaN ... NaN First COVID test Black or African American Male 10003752.0 NaN NaN 606.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
14 case 007c3ba2-630f-4347-b3d9-7612eb4fc2f6 Open-A1 10000364-6585760 73.0 NaN 10000364-6585760 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 272.0 f7bb0d11-a6e8-4504-bc3b-e57bf164e633 ACR_20211020
15 case 007cafb8-437d-4e1e-bbea-71b4912d1ba7 Open-A1 10003752-IsAEaMll3EOuDHg9zbiXhA 54.0 NaN NaN NaN Yes NaN ... NaN First COVID test White Female 10003752.0 NaN NaN 0.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
16 case 008db922-2329-47cb-b50b-003176e65464 Open-A1 10000364-686273 35.0 NaN 10000364-686273 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 272.0 14b6b707-15a8-4379-bdca-b7d9eea1b35e ACR_20210823
17 case 009575e7-6c35-44e5-8435-e4b72205d416 Open-A1 10000364-1102002 78.0 NaN NaN NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 270.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
18 case 00a6a45b-d261-44fb-9eae-87adb7ac6933 Open-A1 10000364-2697320 64.0 NaN 10000364-2697320 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 272.0 2d761222-3669-4a77-9473-342c369fcc40,f670ecfb-... ACR_20211115,ACR_20210827
19 case 00a7e1ef-3358-42bc-91bb-e01569f12623 Open-A1 10000364-2366451 40.0 NaN 10000364-2366451 NaN Yes NaN ... NaN First COVID test Black or African American Female 10000364.0 NaN NaN 274.0 f670ecfb-bc34-4c2d-b15d-4c2762268904,2d761222-... ACR_20210827,ACR_20211115
20 case 00aa2d23-6eff-47c6-9d09-47a4c0e56807 Open-A1 10000364-968592 83.0 NaN 10000364-968592 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 272.0 f7bb0d11-a6e8-4504-bc3b-e57bf164e633 ACR_20211020
21 case 00af0a97-21ef-4b5f-9dd9-03225968c159 Open-A1 10003752-1pHuLYLiMUWoqcp3B6Nhw 34.0 NaN 10003752-1pHuLYLiMUWoqcp3B6Nhw NaN No NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
22 case 00ce1f51-3c12-479d-a42d-dcdb4a1b6aff Open-A1 10000364-52658 71.0 NaN 10000364-52658 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 272.0 14b6b707-15a8-4379-bdca-b7d9eea1b35e ACR_20210823
23 case 00d68288-b45a-4024-9009-493f941fc8a1 Open-A1 10000364-1295788 56.0 NaN NaN NaN No NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 0.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
24 case 00d7bc1b-fbe7-440b-90b2-5f6462b3b501 Open-A1 10000364-6535890 34.0 NaN 10000364-6535890 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 272.0 f670ecfb-bc34-4c2d-b15d-4c2762268904 ACR_20210827
25 case 00dde7cf-97b6-4634-9be8-fc524f913689 Open-A1 10000364-5233414 52.0 NaN NaN NaN No NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 0.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
26 case 00df5184-0af6-415c-ba59-f44dae263f2b Open-A1 10000364-1450498 64.0 NaN NaN NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 286.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
27 case 00e00391-70f9-4279-9826-d5d94f17409a Open-A1 10000364-5622948 70.0 NaN 10000364-5622948 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 0.0 14b6b707-15a8-4379-bdca-b7d9eea1b35e ACR_20210823
28 case 00e0618d-b638-4d81-b871-157840d1865c Open-A1 10000364-1563325 50.0 NaN NaN NaN No NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 272.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
29 case 00ea40da-b44c-42fa-83dc-c70953bb3506 Open-A1 10000364-5435710 71.0 NaN 10000364-5435710 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 272.0 f670ecfb-bc34-4c2d-b15d-4c2762268904,f7bb0d11-... ACR_20210827,ACR_20211020
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
7504 case ff4e5bae-88c4-47ec-a42b-a6abd1f77c0a Open-A1 10003752-IOOvSLTyE6c0aZvKMx5jQ 1.0 NaN 10003752-IOOvSLTyE6c0aZvKMx5jQ NaN Yes NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7505 case ff513b74-c6e1-4b9c-951d-0431b05ccac8 Open-A1 10000364-2617430 57.0 NaN 10000364-2617430 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 272.0 f670ecfb-bc34-4c2d-b15d-4c2762268904 ACR_20210827
7506 case ff52abdc-8b71-4066-a921-a226ec93925b Open-A1 10000364-6560902 55.0 NaN NaN NaN Yes NaN ... NaN First COVID test Black or African American Male 10000364.0 NaN NaN 272.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
7507 case ff558b07-ce01-4ec4-8fae-a3c4e931311d Open-A1 10003752-GhZQEAnaw0K2SjyHCSRLrw 61.0 NaN 10003752-GhZQEAnaw0K2SjyHCSRLrw NaN No NaN ... NaN First COVID test White Male 10003752.0 NaN NaN 0.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7508 case ff626041-5fd9-4164-9705-cf31dd7526f6 Open-A1 10000364-2012729 57.0 NaN NaN NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 272.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
7509 case ff6f794f-91ea-4de0-946a-3d4f3fcaba6d Open-A1 10000364-2077961 7.0 NaN NaN NaN No NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 0.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
7510 case ff72faf5-f7e9-4a31-ba4a-9baa81b4a66c Open-A1 10008204-znWVaidhQEynkQtQ3wMlRA 72.0 NaN 10008204-znWVaidhQEynkQtQ3wMlRA NaN No NaN ... NaN First COVID test White Male 10008204.0 NaN NaN 772.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7511 case ff739871-3a04-4aa2-a86d-11454bda6663 Open-A1 10008204-bevB7abQOEqSfKiF1pFqDQ 66.0 NaN 10008204-bevB7abQOEqSfKiF1pFqDQ NaN No NaN ... NaN First COVID test White Male 10008204.0 NaN NaN 772.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7512 case ff756807-17bc-460e-be5a-f1503105bfeb Open-A1 10003752-Lz796PtOUOgXaTykRH27g 34.0 NaN NaN NaN No NaN ... NaN First COVID test Black or African American Male 10003752.0 NaN NaN 606.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
7513 case ff8b6c96-5e45-4795-a9fd-e24c48833784 Open-A1 10003752-pmO4UKtBXkKXpUJRbpUdnw 1.0 NaN 10003752-pmO4UKtBXkKXpUJRbpUdnw NaN No NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7514 case ff92b2ca-0fd5-4177-90a7-dbdef31cb23d Open-A1 10000364-2468683 64.0 NaN 10000364-2468683 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 271.0 f670ecfb-bc34-4c2d-b15d-4c2762268904 ACR_20210827
7515 case ffa7ec23-bef4-496d-ae82-fbd3796238b6 Open-A1 10003752-6USsNwNCm0iyfIAkbI4Uug 65.0 NaN NaN NaN Yes NaN ... NaN First COVID test Black or African American Male 10003752.0 NaN NaN 606.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
7516 case ffa9b862-e8aa-4525-94ae-247d395946c8 Open-A1 10003752-AcsDqmM60qJQRfqv2gfg 57.0 NaN 10003752-AcsDqmM60qJQRfqv2gfg NaN No NaN ... NaN First COVID test White Female 10003752.0 NaN NaN 604.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7517 case ffb0298c-d87b-4841-b398-a8ea41009b7c Open-A1 10000364-1725993 44.0 NaN NaN NaN No NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 272.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
7518 case ffb042f6-01c9-4b81-98d1-cc55b9d544d8 Open-A1 10000364-6547573 44.0 NaN 10000364-6547573 NaN Yes NaN ... NaN First COVID test Not Reported Male 10000364.0 NaN NaN 286.0 2d761222-3669-4a77-9473-342c369fcc40,f7bb0d11-... ACR_20211115,ACR_20211020
7519 case ffb21de7-7d59-46d3-b497-3788f9f10d42 Open-A1 10000364-1395705 58.0 NaN 10000364-1395705 NaN Yes NaN ... NaN First COVID test Black or African American Female 10000364.0 NaN NaN 271.0 14b6b707-15a8-4379-bdca-b7d9eea1b35e,f7bb0d11-... ACR_20210823,ACR_20211020,ACR_20211115
7520 case ffb420ef-b198-4f93-bca2-07f2c0546e7e Open-A1 10008204-qeRymORTUaySqAH1ry8PQ NaN Yes 10008204-qeRymORTUaySqAH1ry8PQ NaN No NaN ... NaN First COVID test White Male 10008204.0 NaN NaN 0.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7521 case ffb6a26f-fd18-402e-a5c6-57cb50d1601d Open-A1 10000364-1159077 22.0 NaN 10000364-1159077 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 270.0 14b6b707-15a8-4379-bdca-b7d9eea1b35e,2d761222-... ACR_20210823,ACR_20211115
7522 case ffb9a6cb-80ca-431d-aa7d-61c7037be3b7 Open-A1 10003752-DSd2inbH0GOcAxuLlIw 22.0 NaN NaN NaN Yes NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
7523 case ffc56220-524c-44f8-941b-171a619b82a9 Open-A1 10003752-SwYygM5C0aZu7wCMy5qA 53.0 NaN 10003752-SwYygM5C0aZu7wCMy5qA United States of America No NaN ... NaN First COVID test Black or African American Male 10003752.0 NaN NaN 606.0 9bf8adc9-008a-4863-9e8e-bd051555eea6 ACR_20210915
7524 case ffcc96e6-380e-4b61-a317-018e24efce0c Open-A1 10000364-227582 76.0 NaN 10000364-227582 NaN Yes NaN ... NaN First COVID test Black or African American Male 10000364.0 NaN NaN 271.0 2d761222-3669-4a77-9473-342c369fcc40,f7bb0d11-... ACR_20211115,ACR_20211020,ACR_20210827
7525 case ffce4d8c-b684-4edf-80a5-e7e37845fb1f Open-A1 10000364-287258 86.0 NaN 10000364-287258 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 0.0 14b6b707-15a8-4379-bdca-b7d9eea1b35e,f7bb0d11-... ACR_20210823,ACR_20211020
7526 case ffcfe964-b9e2-4ce7-8946-b72138c97fc3 Open-A1 10000364-1664240 18.0 NaN 10000364-1664240 NaN Yes NaN ... NaN First COVID test White Female 10000364.0 NaN NaN 272.0 013383a2-32ca-4c50-97bb-cfd281ea1556,f7bb0d11-... ACR_20210804,ACR_20211020
7527 case ffd6130e-c2fb-4f25-92a5-27288ec331d9 Open-A1 10000364-1260001 61.0 NaN 10000364-1260001 NaN Yes NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 0.0 f7bb0d11-a6e8-4504-bc3b-e57bf164e633,14b6b707-... ACR_20211020,ACR_20210823
7528 case ffe2ad86-0c31-4d1a-a7a1-48a5934e1a06 Open-A1 10003752-Xbd0tHsXEqknBoX4N5HA 58.0 NaN 10003752-Xbd0tHsXEqknBoX4N5HA United States of America No NaN ... NaN First COVID test Black or African American Male 10003752.0 NaN NaN 606.0 9bf8adc9-008a-4863-9e8e-bd051555eea6 ACR_20210915
7529 case ffe6718c-8f07-44cb-985e-633c2a260071 Open-A1 10003752-tUj4JW5dIkKKhf6w9GUCqw 57.0 NaN 10003752-tUj4JW5dIkKKhf6w9GUCqw NaN No NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 604.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7530 case ffe96970-8a14-48fc-8cd1-2fffd305c948 Open-A1 10003752-CCj8Obz5c0q1o2Amtfluw 65.0 NaN 10003752-CCj8Obz5c0q1o2Amtfluw NaN Not Reported NaN ... NaN First COVID test Black or African American Male 10003752.0 NaN NaN 606.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107
7531 case ffece2b4-eb81-4f1a-960c-dd18d96feb3b Open-A1 10000364-940749 67.0 NaN NaN NaN No NaN ... NaN First COVID test White Male 10000364.0 NaN NaN 272.0 2d761222-3669-4a77-9473-342c369fcc40 ACR_20211115
7532 case fff76aad-25ca-4d06-9da5-abd0a18acb42 Open-A1 10003752-PUN2Iozze0iuaH96b6xS2g 74.0 NaN 10003752-PUN2Iozze0iuaH96b6xS2g United States of America Yes NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 9bf8adc9-008a-4863-9e8e-bd051555eea6 ACR_20210915
7533 case fffb8626-c305-4e66-9744-ab8c7eba88c3 Open-A1 10003752-1xOeCuwEj0O1PNBpK90HBg 55.0 NaN 10003752-1xOeCuwEj0O1PNBpK90HBg NaN Yes NaN ... NaN First COVID test Black or African American Female 10003752.0 NaN NaN 606.0 b1487107-9f44-40bd-b1bf-4144415db825 ACR_20220107

7534 rows × 22 columns

In [16]:
## One can then use standard tools in any programming language to do cohort building. 
## Here I'm using the "pandas" Python package to select a cohort based on demographic information stored in the case node.
cohort = list(df.loc[(df['sex']=='Female') & (df['race']=="Black or African American") & (df['age_at_index']>79)]['submitter_id'])
display(len(cohort))
cohort
140
Out[16]:
['10000364-1261658',
 '10003752-WGPvINkkL0CIrSueIVq1Jg',
 '10003752-uXv1gUN50k6mv7tUjnA4iA',
 '10000364-6412754',
 '10000364-1445371',
 '10003752-fgcG89lFo029tEOaehRqg',
 '10003752-Qusqy0HUn0ebZ0uNUpPZ4Q',
 '10003752-nTeM3GIJOkSAtdqFYGZM0w',
 '10003752-DM1ytb31mESVcGstOv6pg',
 '10003752-7RqcKWJlcUOyqHQQXQdKg',
 '10003752-SIQrULeYu0aa2wj0Vlc8ZQ',
 '10003752-dnp2GNsIYEqYNg0Tz0LNuA',
 '10000364-1261808',
 '10000364-2080600',
 '10000364-1380451',
 '10000364-1749434',
 '10003752-WRHmgFHch00GU4s3Cn1SA',
 '10003752-yLzAgX2qDE69PIuW4djGSQ',
 '10003752-Hfy4w9COeEuCM6LR4gwnbw',
 '10003752-Z2wiaR2e4EqPqBQcWBNp9w',
 '10003752-LdWIbKc4CkGdL3JbZApXiQ',
 '10003752-yCjv2IrU5kqAI05tltP3xQ',
 '10003752-JsWe1KAEumMRVwpm5LpQ',
 '10003752-TForj4izLkqZ5W8bsPlY5Q',
 '10003752-z4uukcRJ7EazThNHZYsvCw',
 '10003752-jYqP2dXmUuXzdgQTpTeg',
 '10003752-IptE89okUqdFb6cysr0hA',
 '10000364-1350445',
 '10003752-fPrVyUm40q14GE1kfoxA',
 '10003752-ir82OJFQcUK2JMzcDpBRQ',
 '10003752-sUmpJYfSokqMel7Q5dzQ',
 '10003752-PPOeCrjLe0ymDrbWOtI80Q',
 '10000364-66216',
 '10000364-1485801',
 '10003752-jTxE4cap6EeoE9DZgexRoA',
 '10008204-937Ivt3Bukq2hPT20U0A',
 '10000364-1432396',
 '10000364-1319431',
 '10003752-dYN3RzoP4kYJ7IhSsJhxA',
 '10003752-oPmRWCiKlUM6SyI05jBzQ',
 '10003752-jz386ZCaBUCRBMd262mGw',
 '10003752-At0EiGZ0JkqnduLqYbISpQ',
 '10000364-753841',
 '10003752-UVlHTAzz90OIZkykgRf9qQ',
 '10008204-RzyiAtJEE0ybFAuCqBkSag',
 '10003752-vCqvbFzCL0ijfOGYrB3w',
 '10000364-1003942',
 '10003752-gvGnlL3qL0CB4OWna9gjhA',
 '10003752-HdnjV8oBl0u2P7u9XVAD1Q',
 '10003752-XIzlPNz6Ui86IJnRBvY7w',
 '10003752-5LP7Wxj7CU2dtmo0pWJTMQ',
 '10003752-MPNLeMuMEKO0DnDGbtuQ',
 '10003752-BRjhzfVUa0unnKEp9MeyAA',
 '10003752-QDsGIDWek679273MCmWw',
 '10003752-Ym1LjZz1cEijLfQ0INprg',
 '10003752-SWKaXttpsEnxEtbO8aqXQ',
 '10000364-1249111',
 '10000364-1490659',
 '10000364-1922817',
 '10003752-sh69V58TQkafSp4b8xTdw',
 '10003752-5dNpsur27UGmcfa1A8fAQ',
 '10000364-6583717',
 '10003752-VXhEI7XwkSauPkTg0diMA',
 '10003752-6Ok8ONHvoEG4TyLs2c0LoA',
 '10003752-TghwSAwynUuRjOM7kxMWqA',
 '10003752-6Qs4rF7Zy0GzC77zzUuoQ',
 '10000364-6492944',
 '10003752-oQyapz0pEUmICtml3YcA5g',
 '10003752-eEZ6R7qriUu9nJ2bnmhthA',
 '10003752-o8ZyzbXNtU43dqzRgwlxw',
 '10000364-1598653',
 '10000364-2332757',
 '10000364-6147957',
 '10003752-sH18SMeup0SImCsCs5M2w',
 '10000364-1269962',
 '10003752-B9E1yD3dkul5jBk3jfTMQ',
 '10003752-XSGWHfP64ke0S8lqGv9Ivw',
 '10003752-09fiBUBxMk6YLQuKoepkiw',
 '10003752-9fVajlRxk06kNBV8GCiR6g',
 '10003752-tcBj5Hq6d0rJPbI7jfRgQ',
 '10003752-J7bXYK47Hk2mg47cgct5Q',
 '10003752-iV4weqkDSEGmb4tn45PA3g',
 '10003752-mwGTftPyVUWKA4iAhDEXHA',
 '10003752-8XSbC2KAVEe4biU6gHddw',
 '10003752-lNHy745ppk69nUewxH3HLw',
 '10003752-jyGUC2nO0qyBrPawC0cA',
 '10003752-y18v3iRKDEXZxTwPwfFvg',
 '10003752-LcEO99aflUWiQN4qSLjtuQ',
 '10003752-BKfPKloXREKO9oKvK0pSbg',
 '10003752-RtYN7HC5tUS8QbKrF1guA',
 '10003752-1rPTPcDyl0SVcAMyX3bYzQ',
 '10003752-MQ86qIMjd0a2H42jNN2YA',
 '10003752-4XTm7tLCnU1gzIYioyPXA',
 '10000364-5553322',
 '10003752-jO8b6bPXckisXqTs0rlvQ',
 '10003752-JLJvPS78pUeVD5AF3M1BJA',
 '10003752-LjN5UD29KE6UXIPjxQ4Mw',
 '10003752-2AhUaYloUOZWzvsNodoYA',
 '10003752-mxX4pmrtBk2Yv1QR9IajQQ',
 '10003752-ch9Vm94bE2CGfmLnoQmA',
 '10003752-ss2G9WL7DUOeA6r2qd7CA',
 '10003752-0gvxzsNJCEeLeg1ybdpfBw',
 '10003752-DJjzFJ5eS0uIB0LsRBZpA',
 '10003752-FnSyQJP91UyeitNyCdO1cA',
 '10003752-D5GyK3vzHE2Y8gkZOxEbA',
 '10003752-8MuYrnqyknzcTd85BNZQ',
 '10003752-zhLiJJUEYkqDCFUv6vil0w',
 '10003752-9oEEzVxJtUhU1aSCMfuXQ',
 '10003752-ntPzIWC4UKLsesUICkUw',
 '10003752-Eh56MA4uRE6NANpGgeaew',
 '10003752-pzA7R5XykLNc7gRuyimw',
 '10003752-qyqocYZt7EmEpSbqntCeg',
 '10000364-1413071',
 '10000364-1527925',
 '10003752-NWxjjJx1pES6rP3ay0pCdg',
 '10003752-7gFtp2Y40CUVI5ETRfZmg',
 '10003752-EzFS00HLZ0mtSItnQheJKg',
 '10000364-1428433',
 '10003752-ZtsKSDWSwkqVWEYBpUbaGQ',
 '10000364-1001256',
 '10003752-1EQpytjB1keaOPXT9h1RYQ',
 '10003752-2Jll7aVi20mcmetfi8xo3g',
 '10003752-ZvJaktLHXEGoLkoRVzsZGg',
 '10003752-ubcCCmr0L0ifbHdmg43MJw',
 '10000364-1651160',
 '10003752-GbBwN8Nck0ednXTexEpRGQ',
 '10003752-4gg4ygaew0meTjuz1Mw',
 '10003752-VRQ11303106QWpJpeEQDCA',
 '10003752-opt4ty6FiEScMPNhSh13Xg',
 '10003752-8NkCyxAx0W4beO3Liivyg',
 '10003752-SLyYM4IxEu5L9UAhVednw',
 '10000364-820820',
 '10000364-1335833',
 '10003752-Yi5HggcKGEUDO4so07PQ',
 '10003752-CJ2LiofUqNSCrOvrKy2A',
 '10003752-Yvr5gA17Qkm6C3Nr6ioNKQ',
 '10003752-51S1voqe2UqgRfAdngn5kw',
 '10003752-dE3qiYHJ0WF8WIQtj79lA',
 '10003752-6b3wGi34EmpXamCHL9Z9g',
 '10003752-FulUjEunx0mjzokYb3LR9w']

Using a Python wrapper to get all the data in a particular node¶


  • I've written a wrapper script called Gen3Expansion.get_node_tsvs() that uses the Gen3Submission.export_node() function to export the same node across all projects you have access to in the data commons and then merges the results into a single master table for that node: https://github.com/cgmeyer/gen3sdk-python/blob/5fd6b868374f622221c0c0173a0d9489b190facd/expansion/expansion.py#L219
In [17]:
cases = exp.get_node_tsvs(node='case')
display(cases)
File previously downloaded.
node_tsvs/case_tsvs/Open-R1_case.tsv has 3508 records.
File previously downloaded.
node_tsvs/case_tsvs/Open-A1_case.tsv has 7534 records.
File previously downloaded.
node_tsvs/case_tsvs/Open-A1_PETAL_REDCORAL_case.tsv has 857 records.
length of all dfs: 11899
Master node TSV with 11899 total records written to master_case.tsv.
type id project_id submitter_id age_at_index case_ids country_of_origin covid19_positive data_submission_guid ethnicity ... icu_indicator index_event race sex site_id treatment_info ventilator_indicator zip datasets.id datasets.submitter_id
0 case 00190c16-38d5-4156-b95e-38bbbb5f501b Open-R1 419639-011059 79.0 419639-011059 United States of America No NaN Not Hispanic or Latino ... NaN NaN White Male 419639.0 NaN NaN 949.0 5ecc9107-5d51-40e1-bc22-50599ade61bf RSNA_20211214
1 case 0019ec59-9ed7-4e4c-81f9-c268f04564d5 Open-R1 419639-010858 64.0 419639-010858 United States of America No NaN Not Hispanic or Latino ... NaN NaN Not Reported Male 419639.0 NaN NaN 0.0 38a6adc3-59bf-49dd-bbd6-0d2177ad203d RSNA_20220114
2 case 002443fc-a489-499f-8252-1f75b0c40cb1 Open-R1 419639-004048 79.0 419639-004048 United States of America No NaN Not Hispanic or Latino ... NaN NaN White Male 419639.0 NaN NaN 941.0 04e67623-9b88-4755-bc28-5288210b1389 RSNA_20211026
3 case 002b361e-5afe-4ce0-b6c3-61f194d392c2 Open-R1 419639-000012 59.0 419639-000012 United States of America No NaN Not Hispanic or Latino ... NaN NaN Black or African American Male 419639.0 NaN NaN 941.0 9c23f602-8be9-4582-bae9-7026e6f446fd,5ecc9107-... RSNA_20210820,RSNA_20211214
4 case 00305c7f-86dc-4d55-afe1-4655dc1a1f19 Open-R1 419639-010307 73.0 419639-010307 United States of America No NaN Not Hispanic or Latino ... NaN NaN Not Reported Female 419639.0 NaN NaN 0.0 af721fd1-49a7-42d7-9f5f-b5f83d3ef0ee RSNA_20211117
5 case 004f9807-c0db-404a-9f40-30b36ea85361 Open-R1 419639-000076 74.0 419639-000076 United States of America No NaN Hispanic or Latino ... NaN NaN Not Reported Male 419639.0 NaN NaN 941.0 5ecc9107-5d51-40e1-bc22-50599ade61bf RSNA_20211214
6 case 00532213-0169-4aa0-97db-78aece75df31 Open-R1 419639-001531 45.0 419639-001531 United States of America No NaN Not Hispanic or Latino ... NaN NaN Black or African American Male 419639.0 NaN NaN 946.0 6220bbaa-5671-43ab-8b60-f58a52dd8f46,473968d5-... RSNA_20210922,RSNA_20211006
7 case 0061c678-c972-4da5-a73d-49bd6e17d1ed Open-R1 419639-009656 72.0 419639-009656 United States of America No NaN Not Hispanic or Latino ... NaN NaN Other Male 419639.0 NaN NaN 0.0 aa324c95-1b01-46c9-81c1-a0f4b924e5d5,af721fd1-... RSNA_20210902,RSNA_20211117,RSNA_20211006
8 case 0062852e-9ed0-4070-8ee5-5648c0ca53b6 Open-R1 419639-009460 34.0 419639-009460 United States of America No NaN Not Hispanic or Latino ... NaN NaN Asian Male 419639.0 NaN NaN 941.0 af721fd1-49a7-42d7-9f5f-b5f83d3ef0ee,aa324c95-... RSNA_20211117,RSNA_20210902
9 case 006948e8-4e46-409e-a4da-c24609515a1a Open-R1 419639-010398 68.0 419639-010398 United States of America No NaN Not Hispanic or Latino ... NaN NaN Black or African American Female 419639.0 NaN NaN 941.0 af721fd1-49a7-42d7-9f5f-b5f83d3ef0ee RSNA_20211117
10 case 007d2b0b-9e46-4912-a8be-cfa30dff5373 Open-R1 419639-011328 81.0 419639-011328 United States of America Yes NaN Not Hispanic or Latino ... NaN NaN Not Reported Female 419639.0 NaN NaN 0.0 38a6adc3-59bf-49dd-bbd6-0d2177ad203d RSNA_20220114
11 case 00a086ee-471b-4da4-8b65-4baef8bae353 Open-R1 419639-009814 71.0 419639-009814 United States of America No NaN Not Hispanic or Latino ... NaN NaN White Male 419639.0 NaN NaN 941.0 473968d5-82eb-4b37-95d7-31f72c1c188f RSNA_20211006
12 case 00dde440-de60-4763-be99-032b7b7bc3d0 Open-R1 419639-009876 83.0 419639-009876 United States of America No NaN Not Hispanic or Latino ... NaN NaN White Male 419639.0 NaN NaN 941.0 473968d5-82eb-4b37-95d7-31f72c1c188f,04e67623-... RSNA_20211006,RSNA_20211026
13 case 00faf540-e353-4fdc-bd6c-2fd33c953c8d Open-R1 419639-010602 52.0 419639-010602 United States of America No NaN Not Hispanic or Latino ... NaN NaN White Male 419639.0 NaN NaN 941.0 af721fd1-49a7-42d7-9f5f-b5f83d3ef0ee RSNA_20211117
14 case 0117a301-e8de-4dd3-9b4d-a418b59a9996 Open-R1 419639-003133 83.0 419639-003133 United States of America No NaN Hispanic or Latino ... NaN NaN Not Reported Female 419639.0 NaN NaN 941.0 5ecc9107-5d51-40e1-bc22-50599ade61bf RSNA_20211214
15 case 011b091f-cbe8-4fd8-895c-eae9f93ac2cb Open-R1 419639-008055 85.0 419639-008055 United States of America No NaN Not Hispanic or Latino ... NaN NaN Asian Male 419639.0 NaN NaN 941.0 9c23f602-8be9-4582-bae9-7026e6f446fd RSNA_20210820
16 case 0125631f-f9ff-40d9-b36e-724e90bb9eb0 Open-R1 419639-009749 71.0 419639-009749 United States of America No NaN Not Hispanic or Latino ... NaN NaN White Female 419639.0 NaN NaN 0.0 af721fd1-49a7-42d7-9f5f-b5f83d3ef0ee,aa324c95-... RSNA_20211117,RSNA_20210902
17 case 01410ef5-516e-4aeb-86de-efea06cd7b58 Open-R1 419639-008035 61.0 419639-008035 United States of America No NaN Not Hispanic or Latino ... NaN NaN White Male 419639.0 NaN NaN 946.0 aa324c95-1b01-46c9-81c1-a0f4b924e5d5,af721fd1-... RSNA_20210902,RSNA_20211117,RSNA_20210922
18 case 0142668f-befa-4ec7-a024-c85788e50468 Open-R1 419639-003097 27.0 419639-003097 United States of America No NaN Not Hispanic or Latino ... NaN NaN Other Male 419639.0 NaN NaN 941.0 38a6adc3-59bf-49dd-bbd6-0d2177ad203d RSNA_20220114
19 case 017944c3-0673-4478-900a-b255db0f6f54 Open-R1 419639-001556 81.0 419639-001556 United States of America No NaN Hispanic or Latino ... NaN NaN Not Reported Male 419639.0 NaN NaN 941.0 38a6adc3-59bf-49dd-bbd6-0d2177ad203d,22d66d9d-... RSNA_20220114,RSNA_20220211
20 case 0198754e-b5f3-4e7a-9ace-ace0c2879c56 Open-R1 419639-000196 57.0 419639-000196 United States of America No NaN Not Hispanic or Latino ... NaN NaN White Female 419639.0 NaN NaN 941.0 5ecc9107-5d51-40e1-bc22-50599ade61bf RSNA_20211214
21 case 019ab7ac-2c92-410e-b30a-6c45055cccd4 Open-R1 419639-010216 77.0 419639-010216 United States of America No NaN Not Hispanic or Latino ... NaN NaN Not Reported Female 419639.0 NaN NaN 0.0 af721fd1-49a7-42d7-9f5f-b5f83d3ef0ee RSNA_20211117
22 case 01b054bf-b351-4d61-a910-d29c9ecdc955 Open-R1 419639-003263 21.0 419639-003263 United States of America No NaN Hispanic or Latino ... NaN NaN Not Reported Female 419639.0 NaN NaN 941.0 5ecc9107-5d51-40e1-bc22-50599ade61bf RSNA_20211214
23 case 01b10b01-77d2-499e-982c-9d6259c1b438 Open-R1 419639-008868 46.0 419639-008868 United States of America No NaN Hispanic or Latino ... NaN NaN Not Reported Male 419639.0 NaN NaN 953.0 6220bbaa-5671-43ab-8b60-f58a52dd8f46 RSNA_20210922
24 case 01c63700-af92-4693-bba3-9d6beee6d365 Open-R1 419639-004558 64.0 419639-004558 United States of America No NaN Hispanic or Latino ... NaN NaN Not Reported Male 419639.0 NaN NaN 0.0 9c23f602-8be9-4582-bae9-7026e6f446fd,af721fd1-... RSNA_20210820,RSNA_20211117,RSNA_20210922,RSNA...
25 case 01c71b7f-df19-4866-8d9b-4b80a9e17232 Open-R1 419639-006827 63.0 419639-006827 United States of America No NaN Not Hispanic or Latino ... NaN NaN Asian Male 419639.0 NaN NaN 941.0 aa324c95-1b01-46c9-81c1-a0f4b924e5d5,af721fd1-... RSNA_20210902,RSNA_20211117
26 case 01e307ca-6754-4767-90be-314ea9a5d8ad Open-R1 419639-010718 80.0 419639-010718 United States of America Yes NaN Not Hispanic or Latino ... NaN NaN White Female 419639.0 NaN NaN 0.0 af721fd1-49a7-42d7-9f5f-b5f83d3ef0ee RSNA_20211117
27 case 01e99ca4-6071-42b9-a416-b5f2f88645bb Open-R1 419639-009565 80.0 419639-009565 United States of America No NaN Not Hispanic or Latino ... NaN NaN Black or African American Male 419639.0 NaN NaN 941.0 9c23f602-8be9-4582-bae9-7026e6f446fd RSNA_20210820
28 case 01f31b93-8c89-4c9d-9063-a67d043cd992 Open-R1 419639-003119 34.0 419639-003119 United States of America Yes NaN Not Hispanic or Latino ... NaN NaN White Female 419639.0 NaN NaN 941.0 5ecc9107-5d51-40e1-bc22-50599ade61bf RSNA_20211214
29 case 01f9ded2-51d2-404c-bdba-5f405206d515 Open-R1 419639-000779 56.0 419639-000779 United States of America No NaN Not Hispanic or Latino ... NaN NaN Asian Female 419639.0 NaN NaN 940.0 be97ed98-dea1-44b1-808a-4d1039972434 RSNA_20220105
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
11869 case f6891891-253a-49cc-9d5b-a2c076966fe5 Open-A1_PETAL_REDCORAL U05-00315 NaN U05-00315 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11870 case f6b22247-1628-4f5b-ae6b-2c5924b245d2 Open-A1_PETAL_REDCORAL U01-01662 NaN U01-01662 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11871 case f70df33d-af11-42af-8300-efd3ce7521ad Open-A1_PETAL_REDCORAL V01-01647 NaN V01-01647 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11872 case f74e8047-a393-49c2-a113-6853017545bd Open-A1_PETAL_REDCORAL A01-00902 NaN A01-00902 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11873 case f78dda1c-ab8a-4b0b-a8fc-ff81c5dfbb46 Open-A1_PETAL_REDCORAL H03-00247 NaN H03-00247 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11874 case f889a407-1a7b-481b-9dde-ef69d39e6aef Open-A1_PETAL_REDCORAL U01-01615 NaN U01-01615 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11875 case f916eee6-109c-4605-8b3b-8a02a8ed1fc2 Open-A1_PETAL_REDCORAL D01-01655 NaN D01-01655 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11876 case f955540c-dd77-42c0-9b50-8222540efb85 Open-A1_PETAL_REDCORAL M01-00901 NaN M01-00901 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11877 case f97a81f3-b6e2-4d1c-a77d-881b0da707ae Open-A1_PETAL_REDCORAL D04-01095 NaN D04-01095 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11878 case f9f7da55-8a6b-460e-8e69-4393e4d58c68 Open-A1_PETAL_REDCORAL D01-01659 NaN D01-01659 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11879 case fa08b6e8-fc27-4426-9ef5-64a3b65fed23 Open-A1_PETAL_REDCORAL B02-01501 NaN B02-01501 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11880 case fa7e183b-0ad2-4df7-84f9-b41c64a831e8 Open-A1_PETAL_REDCORAL H03-00243 NaN H03-00243 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11881 case fa8c00f5-2533-41f2-9897-47a05c7b93d4 Open-A1_PETAL_REDCORAL W01-01339 NaN W01-01339 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11882 case fab09373-5b3f-4822-a630-275fb057b171 Open-A1_PETAL_REDCORAL U01-01636 NaN U01-01636 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11883 case fb0e9858-e70f-4f47-ac34-95ffb6103b94 Open-A1_PETAL_REDCORAL C05-00781 NaN C05-00781 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11884 case fbbd37b1-770d-466d-be8a-c73b61d4cc1a Open-A1_PETAL_REDCORAL A01-00941 NaN A01-00941 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11885 case fbf5ad0d-0548-47da-97d6-886bbd23f809 Open-A1_PETAL_REDCORAL C06-00610 NaN C06-00610 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11886 case fc7a9b5d-a031-4ee8-bea0-5ef38526b787 Open-A1_PETAL_REDCORAL C05-00779 NaN C05-00779 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11887 case fce04c54-6833-4a85-b0be-ac42433dfc6c Open-A1_PETAL_REDCORAL B01-01548 NaN B01-01548 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11888 case fd3244ed-9d58-459a-b56c-a6ba49cb6e18 Open-A1_PETAL_REDCORAL B01-00900 NaN B01-00900 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11889 case fd538a5c-effb-4af0-a944-052f571510a5 Open-A1_PETAL_REDCORAL A01-00934 NaN A01-00934 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11890 case fd910a2a-0429-4a62-b083-50f9d55bccc6 Open-A1_PETAL_REDCORAL M01-00929 NaN M01-00929 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11891 case fdacdbf1-0cab-4a19-bcba-c016b7dc95fe Open-A1_PETAL_REDCORAL W01-01327 NaN W01-01327 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11892 case fe0806c3-80aa-4ee5-86a8-ac6a6f4c1837 Open-A1_PETAL_REDCORAL V02-00314 NaN V02-00314 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11893 case fe93e9de-e8fc-4617-a3c5-6eb0492bac0a Open-A1_PETAL_REDCORAL V02-00299 NaN V02-00299 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11894 case ff254815-f395-4bd7-b83d-397d2922283d Open-A1_PETAL_REDCORAL D04-01079 NaN D04-01079 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11895 case ff4fb4cf-5e5d-44ab-9f29-7d8a33b09c6f Open-A1_PETAL_REDCORAL A01-00893 NaN A01-00893 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11896 case ff5c429c-dd78-48cb-9170-bf0c67c78c50 Open-A1_PETAL_REDCORAL W07-00014 NaN W07-00014 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11897 case ffb5c4cb-d9e2-4f77-8e78-308cb1c2e408 Open-A1_PETAL_REDCORAL B01-01572 NaN B01-01572 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220
11898 case ffc28c74-616e-4722-969e-3a1ccfa4da4d Open-A1_PETAL_REDCORAL M02-01108 NaN M02-01108 NaN NaN NaN NaN ... NaN NaN NaN NaN NaN NaN NaN NaN 72fe76f3-27ea-4d92-b480-ac74d9af197c ACR_PETAL_REDCORAL_20211220

11899 rows × 21 columns

Using a Python wrapper to get all the data in a particular project¶


  • Similar to the above example, I've written a wrapper script called Gen3Expansion.get_project_tsvs() that uses the Gen3Submission.export_node() function to export every node in every project (or a particular project) in the data commons. https://github.com/cgmeyer/gen3sdk-python/blob/5fd6b868374f622221c0c0173a0d9489b190facd/expansion/expansion.py#L298
In [32]:
## This example gets all the data in every node of the data model in the project Open-A1
## If "projects" is not specific, all data across all projects you have access to will be downloaded.
exp.get_project_tsvs(projects='Open-A1')
Downloading 7534 records in node 'case' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_case.tsv
	0 records in node 'condition' of project 'Open-A1'.

Downloading 6 records in node 'core_metadata_collection' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_core_metadata_collection.tsv

Downloading 18814 records in node 'cr_series_file' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_cr_series_file.tsv

Downloading 36657 records in node 'ct_series_file' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_ct_series_file.tsv

Downloading 7 records in node 'dataset' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_dataset.tsv

Downloading 3464 records in node 'dx_series_file' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_dx_series_file.tsv

Downloading 20608 records in node 'imaging_study' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_imaging_study.tsv

Downloading 21113 records in node 'measurement' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_measurement.tsv
	0 records in node 'medication' of project 'Open-A1'.
	0 records in node 'mg_series_file' of project 'Open-A1'.

Downloading 676 records in node 'mr_series_file' of project 'Open-A1'.

Output written to file: project_tsvs/Open-A1_tsvs/Open-A1_mr_series_file.tsv
	0 records in node 'nm_series_file' of project 'Open-A1'.
	0 records in node 'observation' of project 'Open-A1'.
	0 records in node 'procedure' of project 'Open-A1'.
	0 records in node 'pt_series_file' of project 'Open-A1'.
	0 records in node 'rf_series_file' of project 'Open-A1'.
	0 records in node 'supplementary_file' of project 'Open-A1'.
	0 records in node 'us_series_file' of project 'Open-A1'.
	0 records in node 'visit' of project 'Open-A1'.
	0 records in node 'xa_series_file' of project 'Open-A1'.
Out[32]:
'Open-A1_case.tsv\nOpen-A1_core_metadata_collection.tsv\nOpen-A1_cr_series_file.tsv\nOpen-A1_ct_series_file.tsv\nOpen-A1_dataset.tsv\nOpen-A1_dx_series_file.tsv\nOpen-A1_imaging_study.tsv\nOpen-A1_measurement.tsv\nOpen-A1_mr_series_file.tsv\n'
In [34]:
!ls -l project_tsvs/Open-A1_tsvs
total 49076
-rw-r--r-- 1 jovyan users  1776810 Jun 22 21:15 Open-A1_case.tsv
-rw-r--r-- 1 jovyan users     1332 Jun 22 21:15 Open-A1_core_metadata_collection.tsv
-rw-r--r-- 1 jovyan users 12225991 Jun 22 21:15 Open-A1_cr_series_file.tsv
-rw-r--r-- 1 jovyan users 23389777 Jun 22 21:15 Open-A1_ct_series_file.tsv
-rw-r--r-- 1 jovyan users     1216 Jun 22 21:15 Open-A1_dataset.tsv
-rw-r--r-- 1 jovyan users  2186247 Jun 22 21:15 Open-A1_dx_series_file.tsv
-rw-r--r-- 1 jovyan users  6419409 Jun 22 21:15 Open-A1_imaging_study.tsv
-rw-r--r-- 1 jovyan users  3790634 Jun 22 21:15 Open-A1_measurement.tsv
-rw-r--r-- 1 jovyan users   445637 Jun 22 21:15 Open-A1_mr_series_file.tsv
In [35]:
## You can then read in the TSVs of data exported from a node to do cohort building / research
tsv_dir = 'project_tsvs/Open-A1_tsvs'
ct = pd.read_csv("{}/Open-A1_ct_series_file.tsv".format(tsv_dir),sep='\t',dtype=str)
display(ct)
type id project_id submitter_id data_category data_format data_type file_name file_size md5sum ... pixel_spacing series_description series_uid slice_thickness spacing_between_slices storage_urls core_metadata_collections.id core_metadata_collections.submitter_id imaging_studies.id imaging_studies.submitter_id
0 ct_series_file 0000ecf4-0982-44a4-9bf9-3da508aa8254 Open-A1 2.16.840.1.114274.1818.49957484958482183452775... CT DCM DICOM 10003752-FTrypNMZECIjF730AFXfQ/2.16.840.1.1142... 1630002 5aff7eef47137bcd418dcf21b38ac96b ... NaN SCOUT 2.16.840.1.114274.1818.49957484958482183452775... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 25076607-be7f-47d3-816b-dcb5fa7363c7 2.16.840.1.114274.1818.46251060062627182281552...
1 ct_series_file 0001ea29-d145-46b3-b655-fdf54d376561 Open-A1 2.16.840.1.114274.1818.47452056560618671341191... CT DCM DICOM 2.16.840.1.114274.1818.57601668720416007191725... 42499230 910821c0599ea333c4be2de47f183136 ... NaN HELICAL ABD/PELVIS 2.16.840.1.114274.1818.47452056560618671341191... NaN NaN NaN NaN NaN ad885b5b-715b-48c9-a418-907a495de131 2.16.840.1.114274.1818.57601668720416007191725...
2 ct_series_file 00077049-9d86-43be-b0c9-dc4b826c785e Open-A1 2.16.840.1.114274.1818.51148225759453066365970... CT DCM DICOM 10003752-1Pj5lmhqK0G7OJDm8wlilA/2.16.840.1.114... 13520868 af9c12fb02d887bcad79edcf8b89caa2 ... NaN MIP, iDose (2) 2.16.840.1.114274.1818.51148225759453066365970... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 523ad313-380b-43aa-8a5f-aeed4bf51570 2.16.840.1.114274.1818.52869657698410731211802...
3 ct_series_file 0008c3b1-a442-4d0d-939b-556e3cb3abec Open-A1 2.16.840.1.114274.1818.49471856660770992231274... CT DCM DICOM 2.16.840.1.114274.1818.50059223583904198521565... 28610238 320cd3091aa0caa700d1a0a71e254658 ... NaN Stone Study 3.0 3 2.16.840.1.114274.1818.49471856660770992231274... NaN NaN NaN NaN NaN cd0aa457-f676-4888-94d2-438c1c55e1fc 2.16.840.1.114274.1818.50059223583904198521565...
4 ct_series_file 0008dff4-0074-4a4f-bb7d-488d69a57867 Open-A1 2.16.840.1.114274.1818.54441193239846891335001... CT DCM DICOM 2.16.840.1.114274.1818.50606639569211286313615... 177471568 567c35e00d413d4c815d7f2f315b30cf ... NaN THINS 2.16.840.1.114274.1818.54441193239846891335001... NaN NaN NaN NaN NaN 65ba9271-5756-4a71-8fcc-f1cd5836671e 2.16.840.1.114274.1818.50606639569211286313615...
5 ct_series_file 000d4373-2468-4803-ad95-dd535616b9d7 Open-A1 2.16.840.1.114274.1818.52155290829062416411654... CT DCM DICOM 2.16.840.1.114274.1818.53764936885997250101628... 29964882 3240c1e31f49966c720c3ad08987b72d ... NaN CORONAL 2.16.840.1.114274.1818.52155290829062416411654... NaN NaN NaN NaN NaN 54eae512-138c-4570-9586-62c715ce2a54 2.16.840.1.114274.1818.53764936885997250101628...
6 ct_series_file 00181318-8477-45c7-8bdc-a324c1ea9fb6 Open-A1 2.16.840.1.114274.1818.49528130810496449261224... CT DCM DICOM 2.16.840.1.114274.1818.52909850575691355561481... 22380960 ded4bd6a84b10718547adce771139566 ... NaN STONE STUDY THICK SLICES 2.16.840.1.114274.1818.49528130810496449261224... NaN NaN NaN NaN NaN 25884807-c3b8-4ca6-bbb7-91fee879fcca 2.16.840.1.114274.1818.52909850575691355561481...
7 ct_series_file 0018a73d-d0e0-4849-98eb-962c9eabfacf Open-A1 2.16.840.1.114274.1818.55615157599364385421040... CT DCM DICOM 10000364-366240/CT_2.16.840.1.114274.1818.5555... 31455812 522824026385ddc22ecdb6a92017882e ... NaN SAG BODY 2.16.840.1.114274.1818.55615157599364385421040... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 3f94f8f2-1722-489f-a698-e39bc88044ab 2.16.840.1.114274.1818.55551698644672994181218...
8 ct_series_file 001a2fe3-9ef0-4e2d-aeb3-3a47ef5c8cb5 Open-A1 2.16.840.1.114274.1818.50444111535957281371522... CT DCM DICOM 10000364-1429707/2.16.840.1.114274.1818.559175... 20977236 8b47ec344a9e2a2f330a5763e653f820 ... NaN Sagittal10.0 MIP 2.16.840.1.114274.1818.50444111535957281371522... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 c9721d8c-6436-4b54-8621-a712a1845784 2.16.840.1.114274.1818.55917510878621680632489...
9 ct_series_file 001b93e7-91ab-41ea-8738-d1c60a60d25b Open-A1 2.16.840.1.114274.1818.49568246924437508717848... CT DCM DICOM 10000364-1432081/2.16.840.1.114274.1818.530726... 29605876 2f0c1880ed7c04c64e5aa058708d3fa1 ... NaN SAG BODY 2.16.840.1.114274.1818.49568246924437508717848... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 3e4d5680-5302-4b1e-a6f5-84b21f6b1c4b 2.16.840.1.114274.1818.53072664876741205902722...
10 ct_series_file 001cef65-519f-4cad-a0f3-ada2c3b8bd35 Open-A1 2.16.840.1.114274.1818.50783701110696030245437... CT DCM DICOM 10000364-1259756/2.16.840.1.114274.1818.481538... 192122 eb6577c19d468ed618590d6bca18d7cf ... NaN Smart Prep Series 2.16.840.1.114274.1818.50783701110696030245437... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 87e018ca-72ff-42cd-a763-e7f210fe7cec 2.16.840.1.114274.1818.48153856909943962611251...
11 ct_series_file 00230d84-a9a6-4305-b0c9-9954652ce3e0 Open-A1 2.16.840.1.114274.1818.55051502853362002701452... CT DCM DICOM 10003752-v7AOAbI1WkOWxd5w6HVOQ/2.16.840.1.1142... 20680718 b056442a416cc0c2eb31025b6137fce3 ... NaN MinIP, iDose (2) 2.16.840.1.114274.1818.55051502853362002701452... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 d9ae2dbe-1e60-475f-bc5b-8203542b4709 2.16.840.1.114274.1818.52573342776289661131695...
12 ct_series_file 00237df2-b061-48f8-9785-09f9f8397f8a Open-A1 2.16.840.1.114274.1818.55890235514104568851395... CT DCM DICOM 2.16.840.1.114274.1818.57262165786098327381143... 23875966 35fa22105259d99a25d007c5445051d0 ... NaN ART SAG DLIR MED 2.16.840.1.114274.1818.55890235514104568851395... NaN NaN NaN NaN NaN b7481b2a-7332-4b4a-b0db-e757aab3d603 2.16.840.1.114274.1818.57262165786098327381143...
13 ct_series_file 002a7182-3c8a-4513-940a-afbb6551ccdc Open-A1 2.16.840.1.114274.1818.50544969920006524349223... CT DCM DICOM 10000364-1521591/CT_2.16.840.1.114274.1818.510... 13309794 5db4041779d83e481e8981886b335e8b ... NaN CTA MIPS CORONAL 2.16.840.1.114274.1818.50544969920006524349223... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 e1720f15-81a1-4148-9dd8-c1e7bc1d8c62 2.16.840.1.114274.1818.51086294230658947977370...
14 ct_series_file 002bc05f-09e8-4620-aec9-f9c4ae7e547b Open-A1 2.16.840.1.114274.1818.46901483095924786198355... CT DCM DICOM 2.16.840.1.114274.1818.50339192942619654751362... 41874874 111cc68bdfabca661a673c5f416a749e ... NaN SAGITTAL 2.16.840.1.114274.1818.46901483095924786198355... NaN NaN NaN NaN NaN 9c734bab-56a2-4e79-9b62-9086a77b4342 2.16.840.1.114274.1818.50339192942619654751362...
15 ct_series_file 002bd07f-f0a5-45ab-b684-7b2d64df7908 Open-A1 2.16.840.1.114274.1818.53852316720529246871304... CT DCM DICOM 2.16.840.1.114274.1818.57272482713393876731593... 34868058 e0621331b50fdbc1456737bc8a20c411 ... NaN DE AP BLEND With 3.0 Br40 2 F_0.5 2.16.840.1.114274.1818.53852316720529246871304... NaN NaN NaN NaN NaN 3ecc2698-0f88-49f3-b6b9-da1fcfe05941 2.16.840.1.114274.1818.57272482713393876731593...
16 ct_series_file 002da9eb-4fa9-40e4-8914-760045e5a894 Open-A1 2.16.840.1.114274.1818.47788547502293258367012... CT DCM DICOM 2.16.840.1.114274.1818.53858790051352121871692... 31275414 2906a85d1c7609c0e9782b8bfbe025e7 ... NaN coronal abdomen 2.16.840.1.114274.1818.47788547502293258367012... NaN NaN NaN NaN NaN a44ec9d3-3cad-41a1-a78f-0e5052308022 2.16.840.1.114274.1818.53858790051352121871692...
17 ct_series_file 0031222a-5f5e-47b9-ae00-1202b9149a3d Open-A1 2.16.840.1.114274.1818.50646714107291197723664... CT DCM DICOM 2.16.840.1.114274.1818.46352567840725257505915... 2736384 3359174b405a1034e4a18650a1b7b981 ... NaN Smart Prep Series 2.16.840.1.114274.1818.50646714107291197723664... NaN NaN NaN NaN NaN dfe98f60-9dc5-43db-82d0-806d03a45d7e 2.16.840.1.114274.1818.46352567840725257505915...
18 ct_series_file 00333556-f6e3-41d0-8577-15598ae21040 Open-A1 2.16.840.1.114274.1818.53838950720515509061928... CT DCM DICOM 10003752-9fbsIMn6cUyofDlSakKuhg/2.16.840.1.114... 17833978 b498beeb3e804e3cf1f9bf33f84be0f9 ... NaN AXL ART, iDose (2) 2.16.840.1.114274.1818.53838950720515509061928... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 a394afe0-c836-4410-a26a-50855493f770 2.16.840.1.114274.1818.52114092694237991004384...
19 ct_series_file 0033eb67-2371-43a3-a81b-2cba69922167 Open-A1 2.16.840.1.114274.1818.48519652760864609681298... CT DCM DICOM 2.16.840.1.114274.1818.47023052143718685801473... 23850546 e525340baee22aaa6a3f3a268d9f76f4 ... NaN Lung 2.16.840.1.114274.1818.48519652760864609681298... NaN NaN NaN NaN NaN 755ea54a-5b74-420a-8391-dde9e302ff47 2.16.840.1.114274.1818.47023052143718685801473...
20 ct_series_file 00351284-e4a3-42d7-af9d-1fd2b8ff9d99 Open-A1 2.16.840.1.114274.1818.49305487510397925131693... CT DCM DICOM 2.16.840.1.114274.1818.52162114235094682361434... 157263020 eb903f2945ba8416e8c80e3d53bac62f ... NaN top down rib to rib 2.16.840.1.114274.1818.49305487510397925131693... NaN NaN NaN NaN NaN 5f0f86df-6376-4fee-a409-e98dea480039 2.16.840.1.114274.1818.52162114235094682361434...
21 ct_series_file 00398cbf-8255-4287-bf9d-45d11bc189ad Open-A1 2.16.840.1.114274.1818.50830320114023229651721... CT DCM DICOM 10000364-5739783/2.16.840.1.114274.1818.538796... 80507682 fb0152cdc25033e7737b02ad72ad2dfd ... NaN PE 1.0mm Thins 2.16.840.1.114274.1818.50830320114023229651721... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 730245ed-5b94-4fd7-94d3-b7c2c50d09ac 2.16.840.1.114274.1818.53879679040957546491255...
22 ct_series_file 0039f8ec-42a7-4043-b60c-d92f2b2a5431 Open-A1 2.16.840.1.114274.1818.49005225458506140691352... CT DCM DICOM 2.16.840.1.114274.1818.56929896772386057438939... 27302466 2e265c1b6029fde2512149ea4309b629 ... NaN LUNGS iDose (2) 2.16.840.1.114274.1818.49005225458506140691352... NaN NaN NaN NaN NaN db69e3e9-6d1d-4ceb-8bd0-f51e34a80261 2.16.840.1.114274.1818.56929896772386057438939...
23 ct_series_file 003aa6f8-7eaa-4ff1-a1b2-6bbdcd8ed69e Open-A1 2.16.840.1.114274.1818.47795015415414848191488... CT DCM DICOM 2.16.840.1.114274.1818.46638061106650275419844... 41880808 b2b6d6e2f01ec1b7b145bc2819276a8e ... NaN sagittal abdomen 2.16.840.1.114274.1818.47795015415414848191488... NaN NaN NaN NaN NaN 58601920-ef3b-40d6-87bb-ec3b579ef54e 2.16.840.1.114274.1818.46638061106650275419844...
24 ct_series_file 003b41b2-3081-4101-922c-02af368b7964 Open-A1 2.16.840.1.114274.1818.49563997547211370316329... CT DCM DICOM 10000364-5203243/CT_2.16.840.1.114274.1818.538... 34277244 eea88381ecc557feb1401b90b138642e ... NaN Cor 2.16.840.1.114274.1818.49563997547211370316329... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 5136665a-c314-45f7-a858-843cc3b1aa39 2.16.840.1.114274.1818.53864404740166466509189...
25 ct_series_file 003cc676-901d-4d8f-8824-56675b34f9af Open-A1 2.16.840.1.114274.1818.53431952655932754076149... CT DCM DICOM 10003752-l6zXbbK3c068a7a37KyWw/2.16.840.1.1142... 61127472 5f049fb9ba9aa77c50bb57439d5d4098 ... NaN CHEST COR, iDose (2) 2.16.840.1.114274.1818.53431952655932754076149... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 16a365bc-98ad-43c0-9880-c432122003ee 2.16.840.1.114274.1818.49706829240474143967564...
26 ct_series_file 003d7dbe-da0b-4edc-9920-8980aec67c47 Open-A1 2.16.840.1.114274.1818.52708517308744556571563... CT DCM DICOM 10003752-GaNatus5c0eNy1txIytdyA/2.16.840.1.114... 76928242 6df7f757ec4f514986a32504b2682e0a ... NaN HIGH RES, iDose (2) 2.16.840.1.114274.1818.52708517308744556571563... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 e5bb7207-9c44-4f65-9b69-a0f12f0b13b0 2.16.840.1.114274.1818.57276554732880748299032...
27 ct_series_file 00430d03-8ffc-4f3b-8f45-2ecd3f733585 Open-A1 2.16.840.1.114274.1818.57443694093781255711482... CT DCM DICOM 10003752-udJLnHWQUECW4u8VcGosLg/2.16.840.1.114... 1002534 c6e75ce61fea57c76ebd125809a35fda ... NaN SCOUT 2.16.840.1.114274.1818.57443694093781255711482... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 df235fcc-68be-429d-8a3d-3d156883ffde 2.16.840.1.114274.1818.54454184266889817543229...
28 ct_series_file 00435260-cdbe-4299-b675-fd1fd3365ac1 Open-A1 2.16.840.1.114274.1818.56289009156633266231891... CT DCM DICOM 10000364-1067510/CT_2.16.840.1.114274.1818.565... 24097560 7abdf2e23870eed93095e34f203a5ffe ... NaN SAG CHEST 3MM 2.16.840.1.114274.1818.56289009156633266231891... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 2a01973b-0f20-46d2-bbfc-b9e467fb6c97 2.16.840.1.114274.1818.56562285241690637041749...
29 ct_series_file 004386de-23df-4980-b2c1-fb8635215426 Open-A1 2.16.840.1.114274.1818.49487280544236776531575... CT DCM DICOM 10003752-3YdcOrFwskyQo1jJYMUI7w/2.16.840.1.114... 14306320 83c42a41ae0f01f0faaabd4e6d484690 ... NaN MIP, iDose (4) 2.16.840.1.114274.1818.49487280544236776531575... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 a1f00bf8-98b1-488e-b4bc-4375ae7d0019 2.16.840.1.114274.1818.56095822575185594662034...
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
36627 ct_series_file ffd57103-c0ee-45a3-8a7f-b195b4140f81 Open-A1 2.16.840.1.114274.1818.50543575489997508864757... CT DCM DICOM 2.16.840.1.114274.1818.54700166749634173131091... 170000 d5217f0157a88db605d6d6c9c346ec32 ... NaN Topogram 0.6 T20f 2.16.840.1.114274.1818.50543575489997508864757... NaN NaN NaN NaN NaN 0b647d5d-4447-4489-a917-b6d99c0ef8a2 2.16.840.1.114274.1818.54700166749634173131091...
36628 ct_series_file ffd8c93d-4bf7-4cb3-bfc5-f16a38b22618 Open-A1 2.16.840.1.114274.1818.49428524375576773818979... CT DCM DICOM 2.16.840.1.114274.1818.48943829450915622837191... 21278498 1380f29ed0d33bbd4cb67a8ee26ea8b4 ... NaN CORONAL 2.16.840.1.114274.1818.49428524375576773818979... NaN NaN NaN NaN NaN 17a07a00-fd77-4620-8e58-f9363274cb57 2.16.840.1.114274.1818.48943829450915622837191...
36629 ct_series_file ffda357d-3a70-4ef1-9564-a6b34287f44a Open-A1 2.16.840.1.114274.1818.48506951762211287641801... CT DCM DICOM 2.16.840.1.114274.1818.47181537340210687507484... 18377846 ae4b0d4a696a5e0b35497126e8ed8a9c ... NaN AXL iDose (2) 2.16.840.1.114274.1818.48506951762211287641801... NaN NaN NaN NaN NaN d68f3f8f-168a-4fba-884d-5f89456db588 2.16.840.1.114274.1818.47181537340210687507484...
36630 ct_series_file ffdabfbb-0f0b-428e-9401-38a9b98e1d0d Open-A1 2.16.840.1.114274.1818.53976293854785801717680... CT DCM DICOM 2.16.840.1.114274.1818.55068505393176182881436... 31156462 47ca7c3cb462afbf7992837d7b0bd420 ... NaN CAP WITH 5.00 Bl56 S2 Lung 2.16.840.1.114274.1818.53976293854785801717680... NaN NaN NaN NaN NaN 4259a1f6-e293-4331-b85c-2c389b8076d2 2.16.840.1.114274.1818.55068505393176182881436...
36631 ct_series_file ffdac164-4ed8-475b-8b44-d6e66e030e8b Open-A1 2.16.840.1.114274.1818.53164750272845914766785... CT DCM DICOM 10000364-2613055/2.16.840.1.114274.1818.533211... 39913920 a85c1aad0e09d17fff2b0c7dc6beb7e1 ... NaN AX CHEST PE STD +C F_0.5 2.16.840.1.114274.1818.53164750272845914766785... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 519abfb8-de35-433a-ba6e-efa283f92b31 2.16.840.1.114274.1818.53321136013187983131761...
36632 ct_series_file ffdbd2d0-b010-4aad-ac63-be2a0308c561 Open-A1 2.16.840.1.114274.1818.57220952741527326011015... CT DCM DICOM 10003752-FTrypNMZECIjF730AFXfQ/2.16.840.1.1142... 13788924 e20fea4184cf77f8a1d69474fb36691c ... NaN CHEST MiPS 2.16.840.1.114274.1818.57220952741527326011015... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 ec5767eb-607a-4c46-aa4f-794760f181f7 2.16.840.1.114274.1818.52855318792067369166696...
36633 ct_series_file ffdc79f5-5434-4cfa-a6b1-69b7d034f67c Open-A1 2.16.840.1.114274.1818.49998392691251620621543... CT DCM DICOM 10003752-0rkAAluz3EuvIj3aFfpWSw/2.16.840.1.114... 31588304 536eec2f24e29fecfb0cdc0f85979a64 ... NaN VENOUS RECON 2.16.840.1.114274.1818.49998392691251620621543... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 db3a49fb-cac6-4e52-b2fe-b427bef1cba7 2.16.840.1.114274.1818.47978731145840567707608...
36634 ct_series_file ffe00d01-ac4c-4c57-b3b2-578d0f3f8501 Open-A1 2.16.840.1.114274.1818.48606589163784907614417... CT DCM DICOM 2.16.840.1.114274.1818.50243181985210264931432... 46133316 6c32da7a7883dae75788f8e01fd0816c ... NaN Sag 2.16.840.1.114274.1818.48606589163784907614417... NaN NaN NaN NaN NaN 94fa2d03-4af1-4cff-a177-384112979b70 2.16.840.1.114274.1818.50243181985210264931432...
36635 ct_series_file ffe085d8-dabd-43eb-ada1-ba33a0d221ae Open-A1 2.16.840.1.114274.1818.51979215377779423361128... CT DCM DICOM 2.16.840.1.114274.1818.52627481775334240721795... 29986326 114039aad2ad5af1448441dc4a2cde53 ... NaN Routine Chest 2.16.840.1.114274.1818.51979215377779423361128... NaN NaN NaN NaN NaN aa3d141f-b07b-4b68-ae47-4434402327a9 2.16.840.1.114274.1818.52627481775334240721795...
36636 ct_series_file ffe11de4-455f-4be2-90a1-3f015d35ab71 Open-A1 2.16.840.1.114274.1818.50673217044353872091121... CT DCM DICOM 10000364-6101237/CT_2.16.840.1.114274.1818.464... 97322494 2352d0e37386801eac5fd811156c4647 ... NaN 1.25x.625 standard 2.16.840.1.114274.1818.50673217044353872091121... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 c6823abe-a991-4b01-9e90-ac0f5843a315 2.16.840.1.114274.1818.46496126341619658791061...
36637 ct_series_file ffe152e2-5174-4c99-8392-1d5849cae1be Open-A1 2.16.840.1.114274.1818.52314624481119033341148... CT DCM DICOM 10000364-5313739/CT_2.16.840.1.114274.1818.472... 59681018 48ae335479a508c6812ebaa45a38b745 ... NaN Sag 2.16.840.1.114274.1818.52314624481119033341148... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 48e2d00b-deaf-4d90-9851-de46126d43d0 2.16.840.1.114274.1818.47200909681673951717850...
36638 ct_series_file ffe293eb-2e2c-456b-875f-dd444f513d8a Open-A1 2.16.840.1.114274.1818.50929911502023812091367... CT DCM DICOM 2.16.840.1.114274.1818.55417295707526896441117... 7254692 0db2de2aee37689bf041f112417f8e46 ... NaN MIN IP 2.16.840.1.114274.1818.50929911502023812091367... NaN NaN NaN NaN NaN 124d9e9d-4f23-431d-a0fd-e9fc502ff48f 2.16.840.1.114274.1818.55417295707526896441117...
36639 ct_series_file ffe2f056-8166-47ce-b090-379a4bdc1460 Open-A1 2.16.840.1.114274.1818.52424787368047838846273... CT DCM DICOM 2.16.840.1.114274.1818.56478399688908078981571... 27479070 9210503af030a8f93ae8321d0a544927 ... NaN LUNG (CHEST) DLIR MED 2.16.840.1.114274.1818.52424787368047838846273... NaN NaN NaN NaN NaN b63c20a0-af33-4ca3-865a-84f70736808f 2.16.840.1.114274.1818.56478399688908078981571...
36640 ct_series_file ffe440b7-78d6-4acc-ad0d-b2f35c65c5f0 Open-A1 2.16.840.1.114274.1818.52488403809436029421106... CT DCM DICOM 2.16.840.1.114274.1818.47661987095151583219404... 47366168 b42c685cbd7ad9febd30cc2079c32d69 ... NaN UNINFUSED 3.00 Br40 S3 sag 2.16.840.1.114274.1818.52488403809436029421106... NaN NaN NaN NaN NaN 6af103f0-0b25-4134-a208-07403428e3da 2.16.840.1.114274.1818.47661987095151583219404...
36641 ct_series_file ffe6eb1f-7979-439e-82ea-e2941f3fa0e0 Open-A1 2.16.840.1.114274.1818.55529830906975846191111... CT DCM DICOM 2.16.840.1.114274.1818.51125861453561074237686... 42102410 ac1af7df8fb301ab6791538c7c6bd153 ... NaN CTAC 70 DFOV 2.16.840.1.114274.1818.55529830906975846191111... NaN NaN NaN NaN NaN ad0fa5e3-9a30-4b64-837c-c9480783ad7f 2.16.840.1.114274.1818.51125861453561074237686...
36642 ct_series_file ffea39ac-a4d4-40d4-bf9e-5f3779743a87 Open-A1 2.16.840.1.114274.1818.48559459829862204641321... CT DCM DICOM 10003752-HAvsOh7FbEp0Y9hPjtBoQ/2.16.840.1.1142... 46041768 3c7045d96fb1263ae079b5bd460766a4 ... NaN SAG, iDose (2) 2.16.840.1.114274.1818.48559459829862204641321... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 7175d709-abba-41c2-8d53-d951707ceedb 2.16.840.1.114274.1818.53346211232635611789588...
36643 ct_series_file ffea85a1-cf64-4359-9d14-3de91b7f34cd Open-A1 2.16.840.1.114274.1818.47788242157270206265069... CT DCM DICOM 2.16.840.1.114274.1818.57578850952836381612377... 43384468 1e0d70f72a79e94a1573d7a94258825e ... NaN ABD/PELVIS W/O 2.16.840.1.114274.1818.47788242157270206265069... NaN NaN NaN NaN NaN 5cbcfa8d-abd1-436d-89e3-60431682fcfd 2.16.840.1.114274.1818.57578850952836381612377...
36644 ct_series_file ffee10f0-d21f-4a66-8a80-358f2774bc0e Open-A1 2.16.840.1.114274.1818.51170213715360990881107... CT DCM DICOM 10003752-fPHT8uqA0q5qRuHLkYPTA/2.16.840.1.1142... 14770614 b6ff8b3724c2031a92c848f990a1ca00 ... NaN MinIP, iDose (2) 2.16.840.1.114274.1818.51170213715360990881107... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 7308986a-3fd4-44b4-8dc6-5eea0f181153 2.16.840.1.114274.1818.48700906795823839421389...
36645 ct_series_file ffee587f-c7a2-416a-89f5-8446eea5b3a1 Open-A1 2.16.840.1.114274.1818.46456861740571239258465... CT DCM DICOM 2.16.840.1.114274.1818.56018763482169963001332... 37466180 cf29d64165af6fc602f84c5e8b12d255 ... NaN Sag 2.16.840.1.114274.1818.46456861740571239258465... NaN NaN NaN NaN NaN 125b9536-a5b0-4880-956c-a6c88eab7fed 2.16.840.1.114274.1818.56018763482169963001332...
36646 ct_series_file ffee73bf-8987-4b5b-aa1b-3f83d0dcfcbd Open-A1 2.16.840.1.114274.1818.52087799792970052711109... CT DCM DICOM 10000364-6548871/CT_2.16.840.1.114274.1818.568... 905308 b8eddf89742b1994432ca7f26599f0f5 ... NaN SCOUT 2.16.840.1.114274.1818.52087799792970052711109... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 21f0a42a-284b-4616-83cf-bb9f7be347cd 2.16.840.1.114274.1818.56892637373518678841527...
36647 ct_series_file ffef3899-d58f-4861-9760-07c67beed0e4 Open-A1 2.16.840.1.114274.1818.55779880859255759781439... CT DCM DICOM 10000364-1775416/2.16.840.1.114274.1818.519177... 53554194 4a401932a98168bafa404cbe9873e6e3 ... NaN Cor MPR 2.16.840.1.114274.1818.55779880859255759781439... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 a0037c18-a695-4acf-92bf-49d4adc7ca22 2.16.840.1.114274.1818.51917729578450709901644...
36648 ct_series_file fff01d22-2bd5-4185-ab77-d5c4b9ac0d61 Open-A1 2.16.840.1.114274.1818.56517423875777975239789... CT DCM DICOM 2.16.840.1.114274.1818.55481603152522975662430... 35880488 321ee723751efaf2c1ba734d8060347a ... NaN sagittal abdomen 2.16.840.1.114274.1818.56517423875777975239789... NaN NaN NaN NaN NaN 499042e3-2145-4b5f-b894-ac0d7a4af401 2.16.840.1.114274.1818.55481603152522975662430...
36649 ct_series_file fff1ccf3-0917-49f2-adff-3f8b02f735e3 Open-A1 2.16.840.1.114274.1818.51521500690102685029326... CT DCM DICOM 10000364-650687/2.16.840.1.114274.1818.5101995... 27824712 56855ea31ffe4bf11eaf0e114febcd37 ... NaN COR BODY 2.16.840.1.114274.1818.51521500690102685029326... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 bc753f0d-da5a-4177-88d4-bbe113e31872 2.16.840.1.114274.1818.51019951011241647966797...
36650 ct_series_file fff3bc2e-a188-4713-899d-85a5d3831519 Open-A1 2.16.840.1.114274.1818.56804893493554909195004... CT DCM DICOM 2.16.840.1.114274.1818.49223169490672576991068... 43249060 05d22aa82da3bee7be2296974a929504 ... NaN Sagittal MPR 3.0 2.16.840.1.114274.1818.56804893493554909195004... NaN NaN NaN NaN NaN d4aaba88-4adc-4de6-8d92-52dee5473f06 2.16.840.1.114274.1818.49223169490672576991068...
36651 ct_series_file fff3cd4a-9929-4668-8e2d-77e7aef8752b Open-A1 2.16.840.1.114274.1818.46130115274371592312353... CT DCM DICOM 2.16.840.1.114274.1818.56931261911713855351136... 23492854 a4095d6067c9fc8f71d5a3dbc8e04b2e ... NaN Coronal 3.0 MPR 2.16.840.1.114274.1818.46130115274371592312353... NaN NaN NaN NaN NaN 2b7aa84c-e515-44a6-ba88-007f34148c70 2.16.840.1.114274.1818.56931261911713855351136...
36652 ct_series_file fff4ac21-8fe4-4ac1-9e41-3c9ee5e1b712 Open-A1 2.16.840.1.114274.1818.49929890207265010191125... CT DCM DICOM 2.16.840.1.114274.1818.52146263260855674077294... 26016868 d04ee65e0bb76ee3921dceeb44f2be43 ... NaN STD AXL 2.16.840.1.114274.1818.49929890207265010191125... NaN NaN NaN NaN NaN 0b8ff751-72e6-4504-b976-5b98130c3476 2.16.840.1.114274.1818.52146263260855674077294...
36653 ct_series_file fff52d87-5c5a-4a11-92f5-f80d6b00862e Open-A1 2.16.840.1.114274.1818.48936021023819044245648... CT DCM DICOM 10000364-1606954/2.16.840.1.114274.1818.524255... 11299930 53c58821a20e1b24629a3e21b08dc20a ... NaN CHEST 70keV 2.16.840.1.114274.1818.48936021023819044245648... NaN NaN NaN e6c5fb12-7467-4a7c-979a-b1e84c5c53df ACR_20210823 820b8580-1dff-4a41-a960-912a42a3de3c 2.16.840.1.114274.1818.52425549090119014511974...
36654 ct_series_file fff8b097-040d-470e-932e-59ead163a32b Open-A1 2.16.840.1.114274.1818.56806844052065594631125... CT DCM DICOM 10000364-6101237/CT_2.16.840.1.114274.1818.464... 22389206 671f6f8e55d3a36ad4bf7219a181dc3b ... NaN chest coronal 2.16.840.1.114274.1818.56806844052065594631125... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 c6823abe-a991-4b01-9e90-ac0f5843a315 2.16.840.1.114274.1818.46496126341619658791061...
36655 ct_series_file fffbd30e-44af-4d1b-b743-ae2de0acd5ee Open-A1 2.16.840.1.114274.1818.50032364059379807521614... CT DCM DICOM 2.16.840.1.114274.1818.54642616033529162091218... 31716034 ff9522be1a76556686f113273a2b1276 ... NaN chest sagital 2.16.840.1.114274.1818.50032364059379807521614... NaN NaN NaN NaN NaN fe61f903-6a51-462d-9e55-0bdad49cd1bb 2.16.840.1.114274.1818.54642616033529162091218...
36656 ct_series_file fffe5fec-e036-4b35-aaa4-1f8c506fbff1 Open-A1 2.16.840.1.114274.1818.48823377996653460923277... CT DCM DICOM 10000364-1052751/CT_2.16.840.1.114274.1818.497... 13196690 86b8ca90f474f62ae2965a4823f34d50 ... NaN Fl_LUNG 1.5 Bl57 2.16.840.1.114274.1818.48823377996653460923277... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 afe6128d-e9ae-4caa-b890-ce4059fb8a82 2.16.840.1.114274.1818.49754138310293527441582...

36657 rows × 34 columns

In [36]:
## Now we can use Python to get the CT series files for the cohort of cases we built earlier
cohort_ct = ct.loc[ct['case_ids'].isin(cohort)]
cohort_ct
Out[36]:
type id project_id submitter_id data_category data_format data_type file_name file_size md5sum ... pixel_spacing series_description series_uid slice_thickness spacing_between_slices storage_urls core_metadata_collections.id core_metadata_collections.submitter_id imaging_studies.id imaging_studies.submitter_id
93 ct_series_file 00b0e0dc-170f-4863-a207-6838fdf1e1f3 Open-A1 2.16.840.1.114274.1818.55964346378096610171168... CT DCM DICOM 2.16.840.1.114274.1818.47546942155557792113367... 13324066 6eb3dcf365021935f8186b1752dbf0e0 ... NaN MIP iDose (2) 2.16.840.1.114274.1818.55964346378096610171168... NaN NaN NaN NaN NaN b1ccf140-c713-49a1-8ed0-e0c9a54bf638 2.16.840.1.114274.1818.47546942155557792113367...
499 ct_series_file 03a16d8b-658b-4fc6-a9eb-9567c96d2e4e Open-A1 2.16.840.1.114274.1818.48959334245767654136969... CT DCM DICOM 10000364-1319431/CT_2.16.840.1.114274.1818.540... 43928694 48e07096e513efa9c7815dfeada1063b ... NaN #PP DE_Thorax 1.5 Qr40 2 B_Sn140kV 2.16.840.1.114274.1818.48959334245767654136969... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 08f71e93-ebc9-4e1e-a4ee-5013e595d3dd 2.16.840.1.114274.1818.54095403294717579441711...
633 ct_series_file 048505b4-e473-43dc-8b65-8fb6ccf9564a Open-A1 2.16.840.1.114274.1818.47761396116332639857774... CT DCM DICOM 10003752-WGPvINkkL0CIrSueIVq1Jg/2.16.840.1.114... 469918 cbee0716c15a9a7f9233550b6647d1b4 ... NaN SURVIEW 2.16.840.1.114274.1818.47761396116332639857774... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 ed36c0cb-d9d9-44c3-a8d2-5cfd0fdaf869 2.16.840.1.114274.1818.51055078868275614981128...
761 ct_series_file 0562f772-8128-454c-8d09-f17cdb5cdb49 Open-A1 2.16.840.1.114274.1818.52447249543864220461728... CT DCM DICOM 10003752-jYqP2dXmUuXzdgQTpTeg/2.16.840.1.11427... 64672416 79eec0a0c0898d6cf425983f78d2c5e5 ... NaN HI RES 2.16.840.1.114274.1818.52447249543864220461728... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 5087e689-35e3-4fb5-93b9-9896d6cf21de 2.16.840.1.114274.1818.48688028118645704643913...
772 ct_series_file 0579f1b2-36de-41f1-bb6e-f12af85c692e Open-A1 2.16.840.1.114274.1818.46261309264012416081379... CT DCM DICOM 10003752-BRjhzfVUa0unnKEp9MeyAA/2.16.840.1.114... 13105028 fbc18badef75b5621fea2003bc92ac9f ... NaN MIP, iDose (2) 2.16.840.1.114274.1818.46261309264012416081379... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 c5f338ef-1e8d-4e46-8cef-290e171d6aba 2.16.840.1.114274.1818.46938907313235410118693...
914 ct_series_file 0672d3ae-27c4-4dfb-b05b-7cc6b4e77aa5 Open-A1 2.16.840.1.114274.1818.49672260042487025331314... CT DCM DICOM 2.16.840.1.114274.1818.52468874831763838151313... 19040976 1d9b8883fa3828539883fd7198cda6a5 ... NaN Chest Sag 2.16.840.1.114274.1818.49672260042487025331314... NaN NaN NaN NaN NaN 586b7fd0-fc8b-4e99-a304-db1fcd959a4d 2.16.840.1.114274.1818.52468874831763838151313...
916 ct_series_file 06752a8f-36d7-45de-92fe-d4e2b18486d1 Open-A1 2.16.840.1.114274.1818.52088358400569405888781... CT DCM DICOM 2.16.840.1.114274.1818.52072902194468673511281... 69860 037df94c1b18afdc211c1d60492bf889 ... NaN Topogram 0.6 T20s 2.16.840.1.114274.1818.52088358400569405888781... NaN NaN NaN NaN NaN 0e457e95-33af-48ee-9cc9-ba5c41098207 2.16.840.1.114274.1818.52072902194468673511281...
1101 ct_series_file 07c44f04-59d7-47ec-a54d-37b02a2f7e72 Open-A1 2.16.840.1.114274.1818.51618659123100181941648... CT DCM DICOM 2.16.840.1.114274.1818.52072902194468673511281... 34612422 d533202bad43e717ff70efaf0e3526c1 ... NaN COR mips 2.16.840.1.114274.1818.51618659123100181941648... NaN NaN NaN NaN NaN 0e457e95-33af-48ee-9cc9-ba5c41098207 2.16.840.1.114274.1818.52072902194468673511281...
1178 ct_series_file 086331a6-c212-4004-94e3-88042e2bf34d Open-A1 2.16.840.1.114274.1818.52789230929689933181196... CT DCM DICOM 10003752-WGPvINkkL0CIrSueIVq1Jg/2.16.840.1.114... 21140310 46c0e664a16934dc8a29fdb1dc482c7d ... NaN AXL, iDose (2) 2.16.840.1.114274.1818.52789230929689933181196... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 ed36c0cb-d9d9-44c3-a8d2-5cfd0fdaf869 2.16.840.1.114274.1818.51055078868275614981128...
1259 ct_series_file 08ee2dd3-193e-433e-b61f-65f8db2d76bc Open-A1 2.16.840.1.114274.1818.47500625575938939955885... CT DCM DICOM 10000364-6492944/CT_2.16.840.1.114274.1818.463... 52298106 0172ef3bdbf53b8e3d5a70db4282fd3e ... NaN Sag MPR 2.16.840.1.114274.1818.47500625575938939955885... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 3b3ecb3d-e4d6-4c13-950a-eec9eea34973 2.16.840.1.114274.1818.46370819974182057886888...
1414 ct_series_file 0a1a54c4-d31a-4172-b0d0-6fc0928aaa60 Open-A1 2.16.840.1.114274.1818.50397103575092183821247... CT DCM DICOM 10000364-1319431/CT_2.16.840.1.114274.1818.540... 48807050 ec7041bb5dfea1573ed0448d09666ffa ... NaN AX CHEST PE STD THIN +C F_0.6 2.16.840.1.114274.1818.50397103575092183821247... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 08f71e93-ebc9-4e1e-a4ee-5013e595d3dd 2.16.840.1.114274.1818.54095403294717579441711...
1415 ct_series_file 0a1c8a10-1197-464c-abb6-733a79cbac23 Open-A1 2.16.840.1.114274.1818.54927525279730560581390... CT DCM DICOM 2.16.840.1.114274.1818.47546942155557792113367... 25982368 24e348db4f0607d2838048bdf0295f91 ... NaN LUNGS iDose (2) 2.16.840.1.114274.1818.54927525279730560581390... NaN NaN NaN NaN NaN b1ccf140-c713-49a1-8ed0-e0c9a54bf638 2.16.840.1.114274.1818.47546942155557792113367...
1514 ct_series_file 0ac1f5ac-948e-484f-abbe-15dfdb3c5ab6 Open-A1 2.16.840.1.114274.1818.52139856230859676691361... CT DCM DICOM 2.16.840.1.114274.1818.47546942155557792113367... 26189378 04cd254193a2add4f14a2ef8fa6bf0a8 ... NaN SAG iDose (2) 2.16.840.1.114274.1818.52139856230859676691361... NaN NaN NaN NaN NaN b1ccf140-c713-49a1-8ed0-e0c9a54bf638 2.16.840.1.114274.1818.47546942155557792113367...
1634 ct_series_file 0b8b4506-9e60-42fa-9abd-a36f37141369 Open-A1 2.16.840.1.114274.1818.55796402103412404711436... CT DCM DICOM 10003752-WGPvINkkL0CIrSueIVq1Jg/2.16.840.1.114... 26328720 69163820da9562541641a9e25b3cf0b8 ... NaN LUNGS, iDose (2) 2.16.840.1.114274.1818.55796402103412404711436... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 ed36c0cb-d9d9-44c3-a8d2-5cfd0fdaf869 2.16.840.1.114274.1818.51055078868275614981128...
1791 ct_series_file 0c99ef80-1fa8-4568-a26d-5a85fbe2f872 Open-A1 2.16.840.1.114274.1818.55152454794275602154480... CT DCM DICOM 10003752-jYqP2dXmUuXzdgQTpTeg/2.16.840.1.11427... 24220430 804d17d27ace478bcb074b83733b9b5a ... NaN VEN SAG CHEST 2.16.840.1.114274.1818.55152454794275602154480... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 ac540136-37c8-47f8-a692-128ba3e3b339 2.16.840.1.114274.1818.47548839393717393311310...
1886 ct_series_file 0d625da5-6509-49e2-b6f0-fa7752653861 Open-A1 2.16.840.1.114274.1818.56531355311072326471027... CT DCM DICOM 10003752-BRjhzfVUa0unnKEp9MeyAA/2.16.840.1.114... 82317034 3a7ca99bf5872c39a5f243e92f5da82f ... NaN HI RES, iDose (2) 2.16.840.1.114274.1818.56531355311072326471027... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 c5f338ef-1e8d-4e46-8cef-290e171d6aba 2.16.840.1.114274.1818.46938907313235410118693...
1923 ct_series_file 0d9c8ac3-95f4-434d-b55f-3f34b399655e Open-A1 2.16.840.1.114274.1818.52546362306442159622974... CT DCM DICOM 10003752-jYqP2dXmUuXzdgQTpTeg/2.16.840.1.11427... 105555128 70cc072ae1eb167fc74dad3cf5d98cfa ... NaN CHEST SOURCE WO 2.16.840.1.114274.1818.52546362306442159622974... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 45dbe3d9-25dd-4bdb-b77f-a4c29ada8259 2.16.840.1.114274.1818.48324760574904930945338...
2035 ct_series_file 0e545904-3ef8-4a82-aa9f-ead20ec91a63 Open-A1 2.16.840.1.114274.1818.47062791267923794261751... CT DCM DICOM 2.16.840.1.114274.1818.50232371834041182001774... 1553584 e3c40e6f456270709e4a5b984a65be8a ... NaN SCOUT 2.16.840.1.114274.1818.47062791267923794261751... NaN NaN NaN NaN NaN c3badb66-676f-43bd-becf-bc58608d3197 2.16.840.1.114274.1818.50232371834041182001774...
2089 ct_series_file 0ed788a2-dce9-429a-b111-21efae900331 Open-A1 2.16.840.1.114274.1818.54346279024422850746055... CT DCM DICOM 2.16.840.1.114274.1818.47546942155557792113367... 20681350 4e79fe1537c3a44d9cc8c41547dd2739 ... NaN AXL iDose (2) 2.16.840.1.114274.1818.54346279024422850746055... NaN NaN NaN NaN NaN b1ccf140-c713-49a1-8ed0-e0c9a54bf638 2.16.840.1.114274.1818.47546942155557792113367...
2201 ct_series_file 0fa5b77e-a71a-4538-a10f-38abc89b6ede Open-A1 2.16.840.1.114274.1818.53418313407643202564340... CT DCM DICOM 2.16.840.1.114274.1818.49162333685696322481506... 32686758 b0b1ab8550fa90a52ec335dffda44390 ... NaN COR iDose (2) 2.16.840.1.114274.1818.53418313407643202564340... NaN NaN NaN NaN NaN 2ab3f394-c819-4fe0-ae9c-cf286ab32b17 2.16.840.1.114274.1818.49162333685696322481506...
2216 ct_series_file 0fbec308-196d-4bec-9081-57cb05600d96 Open-A1 2.16.840.1.114274.1818.56923453410353910621162... CT DCM DICOM 10000364-1749434/CT_2.16.840.1.114274.1818.517... 49938244 fb614691bf3154c9770bb10414535039 ... NaN LUNG 2.16.840.1.114274.1818.56923453410353910621162... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 40ed42e4-d19f-4f3d-bd55-e0a72874bfbd 2.16.840.1.114274.1818.51756275185682361958967...
2327 ct_series_file 1096077e-1cda-44b5-93af-70c977b0b618 Open-A1 2.16.840.1.114274.1818.47450795471739653425584... CT DCM DICOM 2.16.840.1.114274.1818.51339555956390258931766... 237050378 b618c0e0551f3ecc67e685c0be3a21d6 ... NaN AXIAL LUNG B60f 1.5 2.16.840.1.114274.1818.47450795471739653425584... NaN NaN NaN NaN NaN 051a7ef7-9137-4c2c-8b72-7672511d5bdd 2.16.840.1.114274.1818.51339555956390258931766...
2328 ct_series_file 109760b8-d095-4118-8f3b-5a60842e1b01 Open-A1 2.16.840.1.114274.1818.53802420710852412653261... CT DCM DICOM 10003752-jYqP2dXmUuXzdgQTpTeg/2.16.840.1.11427... 18245350 6f4dadfd447c9ccd5a5d9940f5e193c4 ... NaN ART COR CHEST 2.16.840.1.114274.1818.53802420710852412653261... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 45dbe3d9-25dd-4bdb-b77f-a4c29ada8259 2.16.840.1.114274.1818.48324760574904930945338...
2367 ct_series_file 10c06fd3-a6de-4eb4-ae19-7dd679372762 Open-A1 2.16.840.1.114274.1818.54548482228493910589530... CT DCM DICOM 2.16.840.1.114274.1818.50543623976916621593618... 32085606 1fa5ae5626f3356a7769e96b172af340 ... NaN 2x2 cor abd 2.16.840.1.114274.1818.54548482228493910589530... NaN NaN NaN NaN NaN c535df5c-88a8-4328-a8a8-5a7ad0129dd5 2.16.840.1.114274.1818.50543623976916621593618...
2419 ct_series_file 112ca118-40af-4298-aee8-e8f963ff1b1a Open-A1 2.16.840.1.114274.1818.47247454113117857471988... CT DCM DICOM 2.16.840.1.114274.1818.49162333685696322481506... 12195588 222f9da4bda439c4fd78b221c6a3558e ... NaN MIP iDose (2) 2.16.840.1.114274.1818.47247454113117857471988... NaN NaN NaN NaN NaN 2ab3f394-c819-4fe0-ae9c-cf286ab32b17 2.16.840.1.114274.1818.49162333685696322481506...
2491 ct_series_file 11bfed4c-307a-47e3-a60b-b0a2ac454fce Open-A1 2.16.840.1.114274.1818.51827490655301756669271... CT DCM DICOM 2.16.840.1.114274.1818.57135063690911574026567... 34523122 96d368d656cd364690d1ec3941ee2f60 ... NaN Sag 2.16.840.1.114274.1818.51827490655301756669271... NaN NaN NaN NaN NaN 81de2138-d56d-43ee-b79e-4da9bc23828e 2.16.840.1.114274.1818.57135063690911574026567...
2558 ct_series_file 121b72e0-debf-43e0-abd8-314fffb11bc4 Open-A1 2.16.840.1.114274.1818.57532792989429822161041... CT DCM DICOM 10000364-6147957/2.16.840.1.114274.1818.474057... 44255728 651d11a1441e83f9e1059c25398f06f9 ... NaN Sag MPR 2.16.840.1.114274.1818.57532792989429822161041... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 b8e40ac4-e051-4e12-85e7-f96841da3290 2.16.840.1.114274.1818.47405767010972118591122...
2659 ct_series_file 12d2d363-a5b8-4384-bd62-7820a6879223 Open-A1 2.16.840.1.114274.1818.48341341055219706652697... CT DCM DICOM 10000364-1749434/CT_2.16.840.1.114274.1818.562... 40454726 750ac6302c8290a25aab353137133c38 ... NaN AXIALS 2.16.840.1.114274.1818.48341341055219706652697... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 c35746bf-9ca6-4c4b-807d-3d2b7c81663d 2.16.840.1.114274.1818.56236958931622562032480...
2660 ct_series_file 12dc68c2-8a15-4dcf-9d28-03c1933ef108 Open-A1 2.16.840.1.114274.1818.48194208187608996421285... CT DCM DICOM 2.16.840.1.114274.1818.48497422923503503389003... 1293608 d7ed310cb84440da3a196a2d2082987b ... NaN SCOUT 2.16.840.1.114274.1818.48194208187608996421285... NaN NaN NaN NaN NaN 6ddc58c9-ca3e-42d2-9167-655cace7436e 2.16.840.1.114274.1818.48497422923503503389003...
2706 ct_series_file 1332c384-25a6-4573-9703-bcce6026dd38 Open-A1 2.16.840.1.114274.1818.55984719531389594114093... CT DCM DICOM 2.16.840.1.114274.1818.52980787041006160911466... 375404 c4ac6d899529a0306a1ab3a7c5a5deb9 ... NaN SURVIEW 2.16.840.1.114274.1818.55984719531389594114093... NaN NaN NaN NaN NaN 1471390f-a523-4656-9c79-f90c2417472c 2.16.840.1.114274.1818.52980787041006160911466...
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
34288 ct_series_file f02a84e6-86f4-4bcf-86f8-cf1f6c5449cd Open-A1 2.16.840.1.114274.1818.54196933174925358285960... CT DCM DICOM 10000364-1380451/2.16.840.1.114274.1818.576184... 69649710 bedfbf82b37a576c01f3c1b391a8a0f5 ... NaN COR CHEST PE STD MIP +C F_0.6 2.16.840.1.114274.1818.54196933174925358285960... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 d646ae78-6f8c-48e2-b179-242060af43a9 2.16.840.1.114274.1818.57618452499506629833358...
34309 ct_series_file f04c5e73-79d9-43ba-a7d5-b236babd7609 Open-A1 2.16.840.1.114274.1818.56073649327926434991462... CT DCM DICOM 10000364-1319431/CT_2.16.840.1.114274.1818.540... 1863574 414992b03b4af154e4d0b90c16ae0043 ... NaN Monitoring 10.0 Br38 2.16.840.1.114274.1818.56073649327926434991462... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 08f71e93-ebc9-4e1e-a4ee-5013e595d3dd 2.16.840.1.114274.1818.54095403294717579441711...
34483 ct_series_file f16a446a-000f-4149-a1cb-882394aed86d Open-A1 2.16.840.1.114274.1818.49253156750623446381395... CT DCM DICOM 2.16.840.1.114274.1818.55800604469442512806545... 19400978 509f5335362f9eb0592bc5e7eb14cf58 ... NaN Chest Sag 2.16.840.1.114274.1818.49253156750623446381395... NaN NaN NaN NaN NaN a7e7db95-50d4-4c55-bec1-5a61f14b4852 2.16.840.1.114274.1818.55800604469442512806545...
34484 ct_series_file f16ac215-67db-4872-abdd-777501c064ba Open-A1 2.16.840.1.114274.1818.57131787984902380351364... CT DCM DICOM 10000364-1261658/2.16.840.1.114274.1818.515636... 22110098 6799185cb1194332e48965ca345c3e39 ... NaN SAGITTAL MPR 2.16.840.1.114274.1818.57131787984902380351364... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 0ec9de60-e1bf-492c-b7b5-d60bcf5acfd8 2.16.840.1.114274.1818.51563634460029048314588...
34582 ct_series_file f2121ead-5527-4c42-b6a9-e9853315d6a4 Open-A1 2.16.840.1.114274.1818.50719568566687396181626... CT DCM DICOM 10003752-9fVajlRxk06kNBV8GCiR6g/2.16.840.1.114... 17898744 91dccebfa13fe49b55bddca9c3e9f814 ... NaN MinIP, iDose (2) 2.16.840.1.114274.1818.50719568566687396181626... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 d4d5ef53-e937-4c60-a9e0-e6923e3205be 2.16.840.1.114274.1818.49755462096846527711347...
34833 ct_series_file f3b807c8-1733-4425-874c-91b45d6dfedd Open-A1 2.16.840.1.114274.1818.55892374236772729211324... CT DCM DICOM 10003752-LjN5UD29KE6UXIPjxQ4Mw/2.16.840.1.1142... 881474 4a21b109a2c23b616822e27867d8f9c5 ... NaN Scout 2.16.840.1.114274.1818.55892374236772729211324... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 230ce6b6-9d5f-42b4-a93f-5cc21da36a99 2.16.840.1.114274.1818.56722407795165087528672...
34931 ct_series_file f45769a8-db17-4a9e-93d6-5b413b7ad8ac Open-A1 2.16.840.1.114274.1818.54036272945275104811569... CT DCM DICOM 2.16.840.1.114274.1818.52072902194468673511281... 36541080 f9d5e1f71b97aca680f02174b5d0491d ... NaN SAG mips 2.16.840.1.114274.1818.54036272945275104811569... NaN NaN NaN NaN NaN 0e457e95-33af-48ee-9cc9-ba5c41098207 2.16.840.1.114274.1818.52072902194468673511281...
34933 ct_series_file f45a87e3-4402-4a1a-ae8f-a674f69649d5 Open-A1 2.16.840.1.114274.1818.53533151318719223819997... CT DCM DICOM 10000364-6492944/CT_2.16.840.1.114274.1818.463... 30212946 739ec15e6ec83e56c90eeb8c65012966 ... NaN Cor MIP 2.16.840.1.114274.1818.53533151318719223819997... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 3b3ecb3d-e4d6-4c13-950a-eec9eea34973 2.16.840.1.114274.1818.46370819974182057886888...
34966 ct_series_file f48d7064-842f-458e-afe9-8e74df06714f Open-A1 2.16.840.1.114274.1818.57116847183581503751372... CT DCM DICOM 10000364-1749434/CT_2.16.840.1.114274.1818.517... 414750 ee3c672401ec1e3de55a18498b400fa9 ... NaN Smart Prep Series 2.16.840.1.114274.1818.57116847183581503751372... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 40ed42e4-d19f-4f3d-bd55-e0a72874bfbd 2.16.840.1.114274.1818.51756275185682361958967...
35005 ct_series_file f4dbda7b-cd35-41a1-8987-a662e64b4f89 Open-A1 2.16.840.1.114274.1818.52937350291748289925686... CT DCM DICOM 2.16.840.1.114274.1818.52336546464586758891957... 36878704 249b0b3d71f3af03e3111b08377ad0fa ... NaN ABD SAG 2x2 2.16.840.1.114274.1818.52937350291748289925686... NaN NaN NaN NaN NaN cd15c4ec-5c48-47a7-826f-7cadf233198a 2.16.840.1.114274.1818.52336546464586758891957...
35166 ct_series_file f5f9cc4b-7fd0-40c3-b9ae-b38afe98e074 Open-A1 2.16.840.1.114274.1818.48769881958854835438888... CT DCM DICOM 10003752-jYqP2dXmUuXzdgQTpTeg/2.16.840.1.11427... 21938664 7f29c4a6ff3d317a9ccbce881d0430a0 ... NaN VEN COR CHEST 2.16.840.1.114274.1818.48769881958854835438888... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 5087e689-35e3-4fb5-93b9-9896d6cf21de 2.16.840.1.114274.1818.48688028118645704643913...
35180 ct_series_file f6130b7c-fcbc-4f72-bd49-cf850a670384 Open-A1 2.16.840.1.114274.1818.49522960612900783455913... CT DCM DICOM 2.16.840.1.114274.1818.55737110823469461175134... 51936220 dd5be357a284f37e73b057d7e29a1cdb ... NaN HI RES 2.16.840.1.114274.1818.49522960612900783455913... NaN NaN NaN NaN NaN 1a9f00b1-be35-4293-b80c-031c00b78907 2.16.840.1.114274.1818.55737110823469461175134...
35231 ct_series_file f6655af5-4ae5-4aad-bfdb-ee908f35b6aa Open-A1 2.16.840.1.114274.1818.51372862075384294291311... CT DCM DICOM 10003752-dYN3RzoP4kYJ7IhSsJhxA/2.16.840.1.1142... 5804686 a651d7397bbc40f5811a8474d5f07f24 ... NaN STD AXL 2.16.840.1.114274.1818.51372862075384294291311... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 85534fe4-fbb9-4c2e-b6cc-297cc1f6876d 2.16.840.1.114274.1818.55292279166442805687676...
35243 ct_series_file f681be3d-2c48-4a90-9e18-e3b6f7d3e5b7 Open-A1 2.16.840.1.114274.1818.53929921450115012801214... CT DCM DICOM 10000364-6147957/2.16.840.1.114274.1818.474057... 46591636 9dab7e6cada6450e711128e89ff44ac3 ... NaN PE Lungs 2.16.840.1.114274.1818.53929921450115012801214... NaN NaN NaN 5f596eb7-d79b-47d3-8ff0-1d4ba3d2ea85 ACR_20210827 b8e40ac4-e051-4e12-85e7-f96841da3290 2.16.840.1.114274.1818.47405767010972118591122...
35250 ct_series_file f69176f6-9eab-4035-b731-50d31de9cd5f Open-A1 2.16.840.1.114274.1818.54224394692577902551673... CT DCM DICOM 10003752-jYqP2dXmUuXzdgQTpTeg/2.16.840.1.11427... 25202000 593370a0db3bf4ac18beec30762e1caf ... NaN CHEST RECON WO 2.16.840.1.114274.1818.54224394692577902551673... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 5087e689-35e3-4fb5-93b9-9896d6cf21de 2.16.840.1.114274.1818.48688028118645704643913...
35341 ct_series_file f7319fe6-e4de-4a9a-943d-681445782e73 Open-A1 2.16.840.1.114274.1818.57006587110347619497516... CT DCM DICOM 2.16.840.1.114274.1818.52072902194468673511281... 36124288 b082f2467399d512d54f7d383da843e4 ... NaN SAG mips 2.16.840.1.114274.1818.57006587110347619497516... NaN NaN NaN NaN NaN 0e457e95-33af-48ee-9cc9-ba5c41098207 2.16.840.1.114274.1818.52072902194468673511281...
35377 ct_series_file f7919c07-56e2-4bed-a63c-4bd0b07c1d56 Open-A1 2.16.840.1.114274.1818.50354021659339484474916... CT DCM DICOM 2.16.840.1.114274.1818.56770327176963468622571... 77259644 636fec68c654ad6def024f96d90cc446 ... NaN AXL SOURCE iDose (2) 2.16.840.1.114274.1818.50354021659339484474916... NaN NaN NaN NaN NaN a63b1102-304e-4832-8747-d30ed1e4f5c5 2.16.840.1.114274.1818.56770327176963468622571...
35395 ct_series_file f7ad66ab-9f25-4500-9485-b8e0199cf632 Open-A1 2.16.840.1.114274.1818.48227151261486612789842... CT DCM DICOM 10000364-1749434/CT_2.16.840.1.114274.1818.562... 405472 916a4e96b6bc95f72c26c7b0728263da ... NaN Smart Prep Series 2.16.840.1.114274.1818.48227151261486612789842... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 c35746bf-9ca6-4c4b-807d-3d2b7c81663d 2.16.840.1.114274.1818.56236958931622562032480...
35399 ct_series_file f7b21739-6e92-4266-b056-15e6dcc13040 Open-A1 2.16.840.1.114274.1818.49265557179383310379151... CT DCM DICOM 10003752-jYqP2dXmUuXzdgQTpTeg/2.16.840.1.11427... 11595286 ab6aa87685181a09c68f134d9644113d ... NaN ART COR MIP CHEST 2.16.840.1.114274.1818.49265557179383310379151... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 45dbe3d9-25dd-4bdb-b77f-a4c29ada8259 2.16.840.1.114274.1818.48324760574904930945338...
35556 ct_series_file f8b6b008-d7d1-4f2f-b089-c4f2fc7c0662 Open-A1 2.16.840.1.114274.1818.54099736554534158663855... CT DCM DICOM 10000364-6492944/CT_2.16.840.1.114274.1818.463... 36221524 12c7bef1276ace41cd9d1e74e2f48c69 ... NaN PE 2mm 2.16.840.1.114274.1818.54099736554534158663855... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 3b3ecb3d-e4d6-4c13-950a-eec9eea34973 2.16.840.1.114274.1818.46370819974182057886888...
35637 ct_series_file f947bb20-aeb9-42fb-9040-91bcee764058 Open-A1 2.16.840.1.114274.1818.50892074178737076771500... CT DCM DICOM 2.16.840.1.114274.1818.55423042293544820171375... 104404 9d3adccda90845de3aafdb5f43e14bb9 ... NaN Topogram 0.6 T20f 2.16.840.1.114274.1818.50892074178737076771500... NaN NaN NaN NaN NaN 72b01fb1-6384-4c7e-8fff-e927e6960196 2.16.840.1.114274.1818.55423042293544820171375...
35658 ct_series_file f963b907-c150-46a7-bb02-d403cc189a40 Open-A1 2.16.840.1.114274.1818.49632659233623505791789... CT DCM DICOM 2.16.840.1.114274.1818.57135063690911574026567... 16819028 ec785b71199f815aa2dec7ff65da7231 ... NaN A/P W 2.16.840.1.114274.1818.49632659233623505791789... NaN NaN NaN NaN NaN 81de2138-d56d-43ee-b79e-4da9bc23828e 2.16.840.1.114274.1818.57135063690911574026567...
35693 ct_series_file f9a1dae4-d441-4d01-baaf-325e5030a291 Open-A1 2.16.840.1.114274.1818.50800570938328320834464... CT DCM DICOM 2.16.840.1.114274.1818.49959742781222587766338... 20915564 3dfbd5307d5bfe2d5ac1dafdd05b0a75 ... NaN Cor 2.16.840.1.114274.1818.50800570938328320834464... NaN NaN NaN NaN NaN 8d04e616-e3bc-4e21-9091-91c7e6b5eee0 2.16.840.1.114274.1818.49959742781222587766338...
35718 ct_series_file f9cbda0b-c470-46ac-807d-ddf30404caca Open-A1 2.16.840.1.114274.1818.54230946313288245235004... CT DCM DICOM 2.16.840.1.114274.1818.48424175505031677266944... 53514656 ec81b9eb9308e7c720a2dbe39cd052af ... NaN HIGH RES 2.16.840.1.114274.1818.54230946313288245235004... NaN NaN NaN NaN NaN 17b22df1-b216-4520-b4e8-466d30d4925e 2.16.840.1.114274.1818.48424175505031677266944...
35920 ct_series_file fb39040b-489a-4410-88e2-a2e7b3c3c4c0 Open-A1 2.16.840.1.114274.1818.56589161309160872506680... CT DCM DICOM 10000364-1319431/CT_2.16.840.1.114274.1818.540... 191022 6960ab96967c8651dcc728d748767b54 ... NaN TopoBody_lat 1.0 Tr20 2.16.840.1.114274.1818.56589161309160872506680... NaN NaN NaN 23e8c279-92d2-47da-99de-97dedc454bb4 ACR_20211020 08f71e93-ebc9-4e1e-a4ee-5013e595d3dd 2.16.840.1.114274.1818.54095403294717579441711...
36017 ct_series_file fbd5280d-4549-47d3-b636-96808292641b Open-A1 2.16.840.1.114274.1818.54961336870100270459213... CT DCM DICOM 10003752-8NkCyxAx0W4beO3Liivyg/2.16.840.1.1142... 386150 266bbab282a47aa334825a10dc805274 ... NaN SURVIEW 2.16.840.1.114274.1818.54961336870100270459213... NaN NaN NaN 9de1d49d-ad56-44b3-be2e-a2795ce41ff3 ACR_20210915 1920c319-4ac4-4b8f-bb81-d9f6448d89c8 2.16.840.1.114274.1818.56440684053769996992740...
36256 ct_series_file fd2eccbe-55a6-484c-8580-3736acf0afec Open-A1 2.16.840.1.114274.1818.48681936645061680929956... CT DCM DICOM 2.16.840.1.114274.1818.53129077508732444431454... 20340926 2fd3eb2ea5ef52f61a97a720705e56fb ... NaN CORONAL 2.16.840.1.114274.1818.48681936645061680929956... NaN NaN NaN NaN NaN eafbbd76-80d8-4ee3-96f0-3946be17f3ac 2.16.840.1.114274.1818.53129077508732444431454...
36433 ct_series_file fe52abac-ac6b-4eac-a62c-c33c62a45850 Open-A1 2.16.840.1.114274.1818.48439609856238435821718... CT DCM DICOM 2.16.840.1.114274.1818.55737110823469461175134... 1528872 95ac788429830c0cb5f7c7fcb8ea1e4d ... NaN Scout 2.16.840.1.114274.1818.48439609856238435821718... NaN NaN NaN NaN NaN 1a9f00b1-be35-4293-b80c-031c00b78907 2.16.840.1.114274.1818.55737110823469461175134...
36505 ct_series_file fedd02de-c32e-4206-bca9-83cb65ce1146 Open-A1 2.16.840.1.114274.1818.57442183453554543434456... CT DCM DICOM 10003752-WGPvINkkL0CIrSueIVq1Jg/2.16.840.1.114... 7790730 a48d8e5db5e85c3a64f6e84608dc2a3c ... NaN MIN IP 2.16.840.1.114274.1818.57442183453554543434456... NaN NaN NaN 06406925-1d98-4e8f-a0ce-c52b678bcb2a ACR_20220107 f27cc79e-d8b1-4b96-a883-73da2210ce60 2.16.840.1.114274.1818.48752907436280297562336...
36652 ct_series_file fff4ac21-8fe4-4ac1-9e41-3c9ee5e1b712 Open-A1 2.16.840.1.114274.1818.49929890207265010191125... CT DCM DICOM 2.16.840.1.114274.1818.52146263260855674077294... 26016868 d04ee65e0bb76ee3921dceeb44f2be43 ... NaN STD AXL 2.16.840.1.114274.1818.49929890207265010191125... NaN NaN NaN NaN NaN 0b8ff751-72e6-4504-b976-5b98130c3476 2.16.840.1.114274.1818.52146263260855674077294...

482 rows × 34 columns

In [37]:
## You can access the cohort's CT series files by using the 'object_id' field:
object_ids = list(cohort_ct['object_id'])
object_ids
Out[37]:
['dg.MD1R/e6352608-31ae-42fe-9191-69c4ee140a78',
 'dg.MD1R/f0ed36b5-684a-43d6-bb8e-0869f95023a9',
 'dg.MD1R/fb84cfa2-ed9b-4a93-864c-702497e51163',
 'dg.MD1R/faab8935-d248-48d7-9ccc-fa057e695aa2',
 'dg.MD1R/f7b52178-1d0b-4114-80f4-82f3d5b9abb7',
 'dg.MD1R/3a187cec-3f67-4d33-96b6-c92837c426ab',
 'dg.MD1R/690d6e84-e915-4e06-b25a-301c7823077d',
 'dg.MD1R/46f0f909-9c5c-4d3b-87c7-0d98a1c282c9',
 'dg.MD1R/28b3ca44-a883-432c-bb61-ca086ba2a79b',
 'dg.MD1R/555e571c-fc6c-45aa-a8b6-06457516dd5c',
 'dg.MD1R/60c77fe2-935b-4f4d-b087-29edc9f81c18',
 'dg.MD1R/1d620a2a-7dd0-48ef-b345-8d4c11d62c6f',
 'dg.MD1R/2619f74d-ba1b-41db-8fb8-865b321a6930',
 'dg.MD1R/63982efd-422f-4181-9c3f-e13e978bc294',
 'dg.MD1R/58a739af-d6d7-4f18-906f-311ded3fe757',
 'dg.MD1R/8741b3a7-c3a1-4c7f-b691-aa7d22a29202',
 'dg.MD1R/0a340e08-20fd-4c46-9f04-d153b1ba6d96',
 'dg.MD1R/fdce1c3c-4e90-4e61-8aa9-70de114430b5',
 'dg.MD1R/3202f084-c18d-4865-8f3a-6ec368b0d1e6',
 'dg.MD1R/866e8477-26b1-40bc-a475-dc7820b7c669',
 'dg.MD1R/73ef1218-34ef-4183-98f6-57af41933684',
 'dg.MD1R/ff16b6fc-7353-4dab-a8ab-da2ebd1fd789',
 'dg.MD1R/b5af7a00-7abd-4f47-822d-734ced87af20',
 'dg.MD1R/25395b3b-3c8c-415c-afbc-cbf526edaae5',
 'dg.MD1R/cab3a3e8-92d2-4d4b-8f57-cca8226548f9',
 'dg.MD1R/974bb229-4d1f-4aee-8485-14802d3cf890',
 'dg.MD1R/588db577-2f32-405c-88a4-e14f522d27f5',
 'dg.MD1R/32acc276-1ece-49f1-a8ee-257de7043411',
 'dg.MD1R/3379aa45-e3ff-4083-a8ee-1e1088368b19',
 'dg.MD1R/1f2bb55a-2993-49a5-886e-4c66e667bbeb',
 'dg.MD1R/1d5818bb-fd33-4f36-b9d8-5f5eafe5d283',
 'dg.MD1R/ca5d36f4-b134-4ffb-b5ad-b474f7b7bbd3',
 'dg.MD1R/9352692a-0eeb-4b4e-aa4a-0b296a89f592',
 'dg.MD1R/c386196b-edba-41d0-884d-fc3e19cd7930',
 'dg.MD1R/0ef9d051-95ba-4380-af14-25ca8d9a9197',
 'dg.MD1R/70620aba-6eb1-4a28-89ef-49995877c150',
 'dg.MD1R/f6149010-33bb-4b09-85b9-145ba5fe9765',
 'dg.MD1R/fd1d8ef4-9c23-4143-9ce9-f4d9f29aa3d0',
 'dg.MD1R/fd829b7e-4cbc-4ff3-b378-d6ed3b09241d',
 'dg.MD1R/8762b3f5-67e0-46fc-8561-54218bff0d34',
 'dg.MD1R/fe5df2d2-5bd6-4c90-b4ea-47612ce4b4ce',
 'dg.MD1R/46adcf3d-ff76-4d70-96a2-08ff25fcbce2',
 'dg.MD1R/d53fd025-6d99-4173-afe6-26bb7c7fce38',
 'dg.MD1R/d8f829eb-cd51-437f-9071-e19a0feb244a',
 'dg.MD1R/3372adeb-823f-4316-b21f-8ffeb5c0d139',
 'dg.MD1R/3507b91d-8c7e-4c0c-86e0-ec3533c16725',
 'dg.MD1R/7d9a02d1-e5d7-4ec6-9b43-213043d47d57',
 'dg.MD1R/a675a40b-ba3f-473c-bb87-876f8692db4c',
 'dg.MD1R/978ef90d-a5e8-417c-a8b6-6d64f0a409c1',
 'dg.MD1R/54bd9ca9-bf1f-4e96-bf0b-238a2a206d70',
 'dg.MD1R/593e59f1-f5f9-48a7-906c-ecefaa8bdd72',
 'dg.MD1R/0c8760e0-bb5d-4645-8da4-e5d562978b6d',
 'dg.MD1R/5262d211-00d3-421c-912b-a87aa02dbb89',
 'dg.MD1R/a4b6a2d6-8b13-4e44-9152-3afc1974d550',
 'dg.MD1R/fe555323-0fa4-430b-a7e8-9b0ec318a3fa',
 'dg.MD1R/51c99a84-70ff-4068-8873-753b466755a4',
 'dg.MD1R/6fd032fb-17a1-4559-b4de-5a5335205017',
 'dg.MD1R/3040992b-7dc1-4263-baaa-15eeb248a09f',
 'dg.MD1R/1cae17ff-6498-4313-b0e2-ee1318967bae',
 'dg.MD1R/53be8d39-eb99-46ca-b031-66f6106c85cc',
 'dg.MD1R/69dbe438-7139-441b-a405-d05f2d3fe555',
 'dg.MD1R/6ab86ced-14ff-4aa3-b171-cc7428ced8c4',
 'dg.MD1R/515dee39-0442-44e9-be1d-eb0aecb4643a',
 'dg.MD1R/b2b9f0fc-168d-4776-b127-7f1b30c91976',
 'dg.MD1R/bf9ad15d-fd36-4f93-90a4-72c5b3d2fde1',
 'dg.MD1R/ccdea769-43af-44ae-b395-f8b6fb2191eb',
 'dg.MD1R/6dc880fa-0d10-4738-8d00-e9d545d9ddda',
 'dg.MD1R/a3774768-7a48-4e9f-ab18-66cdfbc3eb28',
 'dg.MD1R/77cbd410-b0e9-4c74-b2c5-813d592c73c2',
 'dg.MD1R/d4318db4-775c-4149-8264-efbebde0c8a4',
 'dg.MD1R/92f6b09b-7d25-460e-884f-dc8975d05927',
 'dg.MD1R/1be8bb55-ef23-4f1f-b571-27f6682ac3e6',
 'dg.MD1R/62b15135-02ff-4f44-8715-7261bd8aeaf3',
 'dg.MD1R/7cb38313-46f5-46ef-889d-7b0bae223774',
 'dg.MD1R/2ed7ff67-e474-4602-b1e3-cac65db6cfcc',
 'dg.MD1R/13365139-c90f-4686-8ba7-24c1b9b25786',
 'dg.MD1R/562ea04f-0731-468d-946d-05abcf690cb6',
 'dg.MD1R/9a580c91-d0be-4033-8ee0-72119bb0b1f3',
 'dg.MD1R/7fb883f6-38ca-405a-9cd5-9948c3511ec0',
 'dg.MD1R/7723878c-1b68-4162-9acf-a88cd65879ab',
 'dg.MD1R/57c96097-d93c-4e18-a93a-4f263f125360',
 'dg.MD1R/83b1dba4-0656-4001-91a4-416a28b1d02f',
 'dg.MD1R/d0a504fb-a7fb-49e8-b3b5-87a9dc5c652c',
 'dg.MD1R/756c5206-0da7-4a58-82bb-30a4c2a6beac',
 'dg.MD1R/0bca6ff4-c55a-4988-b702-2a02907942dd',
 'dg.MD1R/ba31905a-ba20-47f5-80ba-e4674f87ad6b',
 'dg.MD1R/df34b966-c1f5-407e-a630-9c642949907b',
 'dg.MD1R/0fecb364-cd5d-4b6b-a3b8-7369a60ca57a',
 'dg.MD1R/da81967f-5a26-42a3-b722-6b853d6e05e4',
 'dg.MD1R/0a70b8ac-c238-4097-bfe4-dcff4adf0551',
 'dg.MD1R/f3ff1a89-939f-40f5-a204-d93dc2d387ff',
 'dg.MD1R/a748ca9c-fd4d-4337-8f65-c66890f71f4b',
 'dg.MD1R/b813ad47-cbfc-42f5-b537-732ed005a0b5',
 'dg.MD1R/f9ad83b9-0ec0-4a22-b565-971bd9e092e7',
 'dg.MD1R/0fd3beb4-ff91-4cd8-a836-017d639b4681',
 'dg.MD1R/dc9502fb-d09f-49de-93b6-5051b2daed23',
 'dg.MD1R/afee1bda-9e66-4df9-9299-dd81ba75cd19',
 'dg.MD1R/1a16045c-7e9e-4383-8156-c76ebf6d1d5c',
 'dg.MD1R/c97e6121-3b82-4bd0-9525-01c52594bdac',
 'dg.MD1R/72d321a9-f9f5-4797-bc3d-6838efd0141b',
 'dg.MD1R/7b2a5695-4ae7-4c6f-b903-f320df84d5f5',
 'dg.MD1R/bac3c24c-7015-4213-97e6-2e9c3761fef9',
 'dg.MD1R/0be7efcf-8aad-4e25-90d7-446bf95db96a',
 'dg.MD1R/09794759-9b92-42fc-bf26-b211e1f34c9e',
 'dg.MD1R/50d8fd24-3ef0-401e-9dea-42cdc3cdd742',
 'dg.MD1R/efb98196-1495-47bd-955a-e7c4cd1cfc9f',
 'dg.MD1R/a24a8c65-d6b1-441e-b241-437519d709b1',
 'dg.MD1R/5ed4babd-5fb5-4c85-9a21-1aa2c848af8e',
 'dg.MD1R/cd70465d-6af9-4fc0-a493-0119972a7039',
 'dg.MD1R/f788af86-0197-454b-b946-70c5acb3b0ad',
 'dg.MD1R/e2070a81-5120-42b2-9839-6365fabe151b',
 'dg.MD1R/c3a84509-361e-49e9-b2b5-27e8b05077fb',
 'dg.MD1R/3b0e403d-004b-448f-8dda-d175a248afa4',
 'dg.MD1R/ea20f1b4-f956-4129-8c57-a6766b173812',
 'dg.MD1R/3bb7a55e-5030-4270-bbd8-764583c8aa20',
 'dg.MD1R/73afccbb-ebce-42b0-ac98-e848bc5b4d01',
 'dg.MD1R/dc0b0db1-5f3a-43ef-9836-69c3fd4500fe',
 'dg.MD1R/c40d52fe-3446-4eed-b033-51fb3e3df4b3',
 'dg.MD1R/97cef358-cad1-42a5-b4dc-22ab1d7399e4',
 'dg.MD1R/befec0c3-8493-44d5-b503-04287278c5a0',
 'dg.MD1R/e806e658-4e52-4de6-8e26-3802c7330425',
 'dg.MD1R/a231f7a7-2b0c-4592-aadf-4ccc35959e92',
 'dg.MD1R/079b68da-5d9b-48cf-a18a-d3e7e45978dc',
 'dg.MD1R/36d05acc-c882-4016-a24a-84d1515dbb55',
 'dg.MD1R/8f0798c6-c817-40b4-ad69-dd4db500b3fb',
 'dg.MD1R/46b1c516-cd38-458a-97de-2408d5e54c22',
 'dg.MD1R/6ec6e7ae-43f3-43c5-b561-89a266588ebb',
 'dg.MD1R/9b40e439-136d-4fb3-9b27-8d3ee7e30e88',
 'dg.MD1R/329c099a-1b6e-434d-81fa-9c75a73764d1',
 'dg.MD1R/4ba43376-4362-43c2-94b2-b3ec2512dbea',
 'dg.MD1R/d0401d29-cbf0-4e55-89f1-68cba158f95f',
 'dg.MD1R/e9780019-b1cd-4b64-990f-76e29365bcb8',
 'dg.MD1R/63850d7e-9432-4219-b452-5458e40b576a',
 'dg.MD1R/f13648c6-61e2-42f4-b14f-e24d36daac44',
 'dg.MD1R/842eeb9d-f910-4c3c-829b-5a50f65d9c7e',
 'dg.MD1R/e963179f-13d2-407e-bda7-da70d1eb00bf',
 'dg.MD1R/82df24fa-6d46-427c-9209-6397cf862d53',
 'dg.MD1R/c0ac7af9-a7d5-48fe-8f50-ad1e36c74465',
 'dg.MD1R/46e66086-8cf2-4241-b040-771e0ad1cc18',
 'dg.MD1R/a493f122-abb4-4310-97b4-dcc28e1aad20',
 'dg.MD1R/58c53b85-7d47-411f-8b05-27c1f1eb11f6',
 'dg.MD1R/30cd03bf-25fe-4b06-9ff9-38ca0290a284',
 'dg.MD1R/63034911-dcab-4e53-baff-f303ea93a7a3',
 'dg.MD1R/9975398c-06dd-46fa-a6c9-6b093a6bfe46',
 'dg.MD1R/155b7388-61d2-4099-91a9-5872ffb614ea',
 'dg.MD1R/71e0dfc8-b18b-4759-b20f-86d72b5c025e',
 'dg.MD1R/e3d9b0c0-08fa-4914-9a8e-72d001e2b27d',
 'dg.MD1R/cdc5b7f9-bf8c-4d72-8380-f2d4e8caaa3e',
 'dg.MD1R/2a19a1df-c607-4b04-b048-3e36f6934b1d',
 'dg.MD1R/b83bb72e-8825-4836-bece-490bc85eb0c3',
 'dg.MD1R/e4a529a2-d83f-4dbe-8768-3aec055d5895',
 'dg.MD1R/7f20f246-626b-4209-9227-af682c9200ef',
 'dg.MD1R/d2f92614-dff4-4063-85fc-d72b901206a0',
 'dg.MD1R/6ab4c646-0497-471b-bba2-cc411cfbe03e',
 'dg.MD1R/c97cb44a-a92c-4fd8-ade3-e6c3c547e5ba',
 'dg.MD1R/bfae42ab-6a23-4937-b14c-dd04d09716ea',
 'dg.MD1R/6ef360e9-31c3-4d3c-8d2e-ea5b22cda803',
 'dg.MD1R/ca5f8a9f-e2d3-4abc-821d-9768588bed63',
 'dg.MD1R/0edacebe-b972-4316-ac0f-da7577642dc1',
 'dg.MD1R/9c43c4ef-7e3b-4289-8e55-8e2360756d7f',
 'dg.MD1R/4ca08962-0a91-4203-83b5-5535bd529090',
 'dg.MD1R/d9258bbe-a541-43e8-8da0-e7786a2d0902',
 'dg.MD1R/9f34ff60-6142-4361-b944-9575620f126a',
 'dg.MD1R/aa2e703a-2d33-4022-8674-751afca1a5b4',
 'dg.MD1R/5452a28c-59d7-4845-b444-dccc9318988d',
 'dg.MD1R/e0aa8ae3-1c06-481d-8ec0-e449c5711d38',
 'dg.MD1R/356493f5-017a-454d-9726-794ac00976b4',
 'dg.MD1R/a53f244b-bffb-4cc5-85a5-bbc86c424b91',
 'dg.MD1R/a6b5d981-d436-470e-b4a3-6e5e78526c10',
 'dg.MD1R/c83ff63a-bc3e-4b3a-9a45-0c165001ee2f',
 'dg.MD1R/1dbc50e1-9321-4eda-958c-8e3265867ca5',
 'dg.MD1R/03cbac84-d327-4bc6-8aef-ac8349a49847',
 'dg.MD1R/c4b463b6-b9fd-41c8-831f-e40c2acf8fb5',
 'dg.MD1R/8cbd3490-247b-4e2d-94c1-4ca978cd6e32',
 'dg.MD1R/454df876-1704-437c-a223-8b42e0146fa7',
 'dg.MD1R/c5e281f5-e633-4c92-95d5-9b8146df1e6a',
 'dg.MD1R/1bb587c6-8e84-4130-af65-36ad1fe90e0c',
 'dg.MD1R/3040d1c0-6aa0-471d-920d-e646715bdd80',
 'dg.MD1R/c53ba2c2-68c7-44ba-94c1-838adae99455',
 'dg.MD1R/ee77b86f-c6be-4d8a-a209-f942debb8d86',
 'dg.MD1R/04979bc8-1187-4f62-97ad-526dac1460b7',
 'dg.MD1R/88de2143-b218-467a-bf62-e43395171ea2',
 'dg.MD1R/8f3c84d8-6f78-4446-bd3a-70a88a4a8e29',
 'dg.MD1R/7701f992-9a68-47c9-8c0c-604a811f5d33',
 'dg.MD1R/e03553b4-8511-4ea2-a6c1-fc4a6949b997',
 'dg.MD1R/55aa4ffb-c3b6-4ff9-83b0-559ea145a8a7',
 'dg.MD1R/46e61b36-3b4c-4cce-9dfb-fc2029a3d07f',
 'dg.MD1R/dc8079c8-6fed-42df-9ab5-5ebe5afc43f0',
 'dg.MD1R/deb979e5-130a-44fe-80b9-150867d254ca',
 'dg.MD1R/c06cb5a6-e8a0-47cf-84d6-25d9fdaddbce',
 'dg.MD1R/0a8102ed-90c7-4d31-bf0c-c79e61ed470a',
 'dg.MD1R/6f4b94cb-8f8c-4979-9fe0-e60ccdf0aec8',
 'dg.MD1R/de2512e8-c810-48ad-8bfa-86794229fcfd',
 'dg.MD1R/6e9082a4-35da-4838-afe7-4bb48ec6b7b5',
 'dg.MD1R/77362063-c331-4823-b857-c80bb0bd64e1',
 'dg.MD1R/6cf5f367-64de-4848-a375-a537f2d248a2',
 'dg.MD1R/61f470ea-0fd2-4efb-bf3a-07a12fe4dd58',
 'dg.MD1R/64a5198b-876e-4507-a240-232a2afc5fd8',
 'dg.MD1R/6c318d85-69c1-4c80-9d51-afea89dc0ea3',
 'dg.MD1R/8aef86b9-54df-468e-b84d-f86d120ea4ed',
 'dg.MD1R/bb5e98b2-7e48-4495-8035-7b755a027a46',
 'dg.MD1R/2dfd0a6b-4522-480b-9461-12d0da62036c',
 'dg.MD1R/84ac4a72-5be3-4de3-85a7-f7a6f5980415',
 'dg.MD1R/ee241ee9-491c-4b27-ab4a-8b203e080805',
 'dg.MD1R/54cd9eeb-aa79-47e8-bdae-410f0efb512e',
 'dg.MD1R/fe5c7296-af1b-466f-9920-a695993b67c6',
 'dg.MD1R/0c183c31-aa76-4af3-a93d-041428c66350',
 'dg.MD1R/e25dd632-df8a-4fbb-ab01-e9f8cf482907',
 'dg.MD1R/18b9b3c9-80c1-431b-977c-64cb93fe5413',
 'dg.MD1R/debe2dcc-cabe-4b22-ac63-1b5dbf8858e4',
 'dg.MD1R/420b4012-1034-4ef7-8935-c8a4d04af8d5',
 'dg.MD1R/1330b345-87d1-4b6b-8db6-26159a6be83d',
 'dg.MD1R/55987478-b7dc-401e-a371-f29bba62c26f',
 'dg.MD1R/6beb4103-2c48-47f0-9b87-2069d051b400',
 'dg.MD1R/6e344262-d423-4b76-9d5c-be329c242e69',
 'dg.MD1R/2d1a631f-5e18-4ba8-a60f-7ddf91ff3247',
 'dg.MD1R/f6e9fe4e-945c-4b88-adb4-47cae4849100',
 'dg.MD1R/9f5fa5b8-82f2-479c-a2d2-e749fbfc6005',
 'dg.MD1R/1ae0e801-e429-4b1d-a5b1-b3d3cfe9c4db',
 'dg.MD1R/8743c42a-717d-401e-8e6d-a33524040b02',
 'dg.MD1R/c22a5ec9-68dd-4d54-8392-5f4e34a79676',
 'dg.MD1R/0cd84907-88ea-4d11-acde-81fc54ce7751',
 'dg.MD1R/717bbaa1-97e9-4e18-b649-8431435eec6c',
 'dg.MD1R/ceb9de1b-cde1-47b2-8034-8b8ab746d45a',
 'dg.MD1R/5c81c579-9785-4d8d-844f-a0c81e9ea96d',
 'dg.MD1R/cc223b53-3278-48d5-8e52-e4aaefb56682',
 'dg.MD1R/f6b68265-423a-4b9a-83bc-5304c0196875',
 'dg.MD1R/5178bd72-a679-4383-8a70-876f6103f3bb',
 'dg.MD1R/33690ce8-8de8-4442-8e5b-dc1ab5490fe7',
 'dg.MD1R/b4f35b07-c4ab-4484-bbf6-ad6ef07976c7',
 'dg.MD1R/35586197-0dd5-417b-9f00-2c882e3ad87e',
 'dg.MD1R/bc250439-d12f-4b38-8b7d-dde8321a36ba',
 'dg.MD1R/6f8578c3-1084-4cca-830e-03ec907a2185',
 'dg.MD1R/14f64ec8-d79b-4b7e-ae33-6282402289ef',
 'dg.MD1R/96a61f2b-960d-4d40-a9b6-86c0cc8f96e2',
 'dg.MD1R/fd289d4e-73fb-466c-a797-9a3af7fedd7d',
 'dg.MD1R/3b8412ce-5769-4976-8092-2a3d50b043c0',
 'dg.MD1R/c821aa54-c7f5-47f6-9731-8365be2486b4',
 'dg.MD1R/5161700d-0c84-4b15-b931-58634c25a9ba',
 'dg.MD1R/54c1bd0b-ce68-4ddc-8767-9862dabe5e3a',
 'dg.MD1R/13826c22-6ce6-4650-92ec-6364ba19e320',
 'dg.MD1R/b90fdec4-a510-4373-bc39-3cd012b3627d',
 'dg.MD1R/73e6e550-30b3-4994-9e32-6a9a9117f8cf',
 'dg.MD1R/be06a6c9-6e0a-49c8-b4a2-e0058966e3a1',
 'dg.MD1R/573bb3bc-83e3-41c4-9228-c37b10d7b694',
 'dg.MD1R/8ed15962-c050-484f-8525-d3d18b726d5d',
 'dg.MD1R/11efa679-3358-4ae3-80ab-884739a26cda',
 'dg.MD1R/16f08475-8f4e-4cae-aaa1-d4b0bc507332',
 'dg.MD1R/b1641d31-58e8-4d00-9c87-f0232e8b884d',
 'dg.MD1R/5ed75b7f-c641-4d04-858c-b4ad280ac6fe',
 'dg.MD1R/9f77d905-50c1-4f49-81e9-88c00f17c871',
 'dg.MD1R/8e6a30b5-8b85-4feb-a312-b63d9b928222',
 'dg.MD1R/54898f12-e1d6-427b-b041-8e8ba6b2efcb',
 'dg.MD1R/9515d3c3-e7dc-4d33-acd7-dcc22b6ed0a8',
 'dg.MD1R/aa0c1923-554f-4417-9660-20a92726a979',
 'dg.MD1R/d3aaf3e5-1274-4c93-bfb6-25a104018de3',
 'dg.MD1R/93606092-b9ca-4dfb-9a5c-af34febca3b8',
 'dg.MD1R/a788771d-080d-4aea-972f-9c8b2230668e',
 'dg.MD1R/1c8941ef-4c1f-4d23-adec-cd9110798e81',
 'dg.MD1R/68aac737-b62b-4b68-9eaa-c2e0452e8881',
 'dg.MD1R/d7445209-abc0-498e-957b-c0d24bd2673f',
 'dg.MD1R/fcdf46ff-37d3-4838-af58-a08ce4956a7f',
 'dg.MD1R/d6fc03cb-2649-4e72-baaa-e4347ab3b2c7',
 'dg.MD1R/e461fe26-d2fa-41ca-b252-2913510de6d1',
 'dg.MD1R/abd9e04a-1ae5-4b6c-948d-f14924480c7e',
 'dg.MD1R/43faef35-fe68-4d0d-85ba-33eb6328c198',
 'dg.MD1R/f4d08ece-7485-47d6-b7ab-23431276866c',
 'dg.MD1R/1582ce82-eb47-40dd-b435-be5d5cc4fb18',
 'dg.MD1R/daff6cb0-acda-4c07-a32a-7b5a36e05772',
 'dg.MD1R/4b7e1a51-0196-4872-9bb1-db993b906c1c',
 'dg.MD1R/58e4ebe8-10b3-44d7-ba8f-7c591a9b8340',
 'dg.MD1R/f39cd056-30a5-4a8d-885b-8144f55e041a',
 'dg.MD1R/ffac39fa-26b3-4385-9c93-09c10b953068',
 'dg.MD1R/d12ec7ad-81cf-43ca-8bfd-ce305d1749cc',
 'dg.MD1R/0d066459-43c3-4938-a4ce-4e3c60acdcd8',
 'dg.MD1R/6c6e3bc7-f97c-4bb4-9eaf-310124cac860',
 'dg.MD1R/c628059e-8aa2-48b8-8ffc-bd3c7b0a02c6',
 'dg.MD1R/258f3a7b-ec91-4ad2-b91d-91466a4c40df',
 'dg.MD1R/2c944d54-90fa-4051-af02-dbc6d5b97fc9',
 'dg.MD1R/1b5faf72-ff1f-4bb7-8488-2d8906d54639',
 'dg.MD1R/20692f47-8420-4c1b-88e9-a97f5436f570',
 'dg.MD1R/84bd3508-465b-4083-8e9a-54feffd4ddfb',
 'dg.MD1R/67c75710-903c-4eb2-a2fc-549ea6f7dc2d',
 'dg.MD1R/addad950-901b-4cd1-a0d8-910357719c27',
 'dg.MD1R/c8979287-ab09-4ae8-a8d9-5d49d12af19e',
 'dg.MD1R/a1ab07ca-5980-463b-8f58-a7b637b83256',
 'dg.MD1R/bda621dd-164d-4a91-ae68-5c8ac691fa51',
 'dg.MD1R/004b2ec6-56d9-44b8-ab47-6d763da12e43',
 'dg.MD1R/e302f9da-d4fa-4b7e-8772-fb94fb465991',
 'dg.MD1R/e9056ebf-9975-46ea-8aff-2f5559fce3da',
 'dg.MD1R/bda5327a-c049-4145-b879-1a658bd56968',
 'dg.MD1R/1a7e5ba7-57d4-4a4a-b124-8aa90a62f716',
 'dg.MD1R/7e7596b1-2251-4ac4-b765-eeb369cd93eb',
 'dg.MD1R/9b6e1e52-1aaf-41c6-a077-a1daf97afc1e',
 'dg.MD1R/818228a6-1314-458c-a51c-753c92dc7e40',
 'dg.MD1R/415f6508-adb8-43be-a8a9-0222bb7bb0d6',
 'dg.MD1R/1c07ea5b-8f9a-4aca-8a5c-03cdf2dd0174',
 'dg.MD1R/9fed28ee-f211-434c-a311-33bf651a541e',
 'dg.MD1R/8c513a83-f96f-4a3b-aa6f-fe9102c4f2de',
 'dg.MD1R/c8871256-1598-4d16-8848-b2b015cfec8f',
 'dg.MD1R/bd9108f2-84ab-448b-871b-da8d5bd730f7',
 'dg.MD1R/64321cca-e054-4cd4-ba0d-8f757b897d09',
 'dg.MD1R/a187a895-1fd2-45d1-8dcd-7f8db59b88bc',
 'dg.MD1R/b192e93d-9ce6-4788-b8e0-be184be958cb',
 'dg.MD1R/34a6157f-2e44-4d87-bcff-b50ed5087919',
 'dg.MD1R/ec9d8f4e-8620-4334-9514-f512f0fd374b',
 'dg.MD1R/a6c3856b-127f-452a-a057-eeb9dae5d1e5',
 'dg.MD1R/65b9137d-3938-4696-aa1e-85c1d6695daf',
 'dg.MD1R/d48a8dea-c408-4d2d-b0bf-70385ccb73c4',
 'dg.MD1R/019e594f-5734-4563-a546-4e194f6e0f60',
 'dg.MD1R/cb0a1902-8f8a-4c24-b6a3-354d040de6ea',
 'dg.MD1R/6e9fa074-34b8-477e-ac3b-c990ed5a2593',
 'dg.MD1R/1fc97bd7-339e-4a58-9e12-c7f78f36c047',
 'dg.MD1R/0c9410ff-5620-4c9c-bd5c-7c0635bbd00a',
 'dg.MD1R/527f55c3-9021-4a50-a8ea-6cdd623ce9a7',
 'dg.MD1R/8fc5836f-f3e6-45aa-a31a-44fbceff9a07',
 'dg.MD1R/b6b981d2-fb9f-403e-b8a0-89ac790e39f8',
 'dg.MD1R/2edda6f2-ba5c-45db-930d-7876d126e19e',
 'dg.MD1R/05a211ad-877e-4a1a-8a60-bd10c88fa698',
 'dg.MD1R/61bcc35a-76d1-43fa-8938-f1c782bf2e8b',
 'dg.MD1R/627074b9-18c2-472a-b713-b81fe4220b37',
 'dg.MD1R/dbb4de13-5ca0-48f5-bc30-a722dbbe74ad',
 'dg.MD1R/ed3c7a23-c0f7-4b8c-959c-0ca3e8e9c3fa',
 'dg.MD1R/f6d65d33-0e05-4636-b9f5-5854752cbf1c',
 'dg.MD1R/7109603c-8d78-45d2-a473-b6947958ccbd',
 'dg.MD1R/aea8dd7d-b925-4dd7-acc5-1dbd91493856',
 'dg.MD1R/4d606cc5-4d02-458e-84f7-7f69a68ba448',
 'dg.MD1R/6a966e23-937a-404f-93e7-3bf3feaee1ea',
 'dg.MD1R/57e9b0f5-7a64-4b8a-8aee-6b976c70c806',
 'dg.MD1R/88b1b26c-e7b5-48e2-a2b7-a05f7bfa528f',
 'dg.MD1R/afa39802-fe03-46c7-b6bd-10c94bffb740',
 'dg.MD1R/deec4133-ce0f-44f7-8334-9f0e581b6f33',
 'dg.MD1R/c4313f5f-b56e-4e3f-8a84-b356d6a6a7c7',
 'dg.MD1R/7cf52d1a-ab0e-48f1-9545-4b750a203b57',
 'dg.MD1R/1a495bfe-16ca-48fb-8a5c-f469ea695e36',
 'dg.MD1R/05fad9c2-5663-4ff0-b5e4-2a087d79789c',
 'dg.MD1R/4a930f11-d20b-4249-9d32-7007b1b75547',
 'dg.MD1R/f7644836-3429-4c04-ab81-ad20676cea35',
 'dg.MD1R/43a22778-a439-463a-93af-bbccaed588b4',
 'dg.MD1R/71099491-30aa-4dfe-9baf-2d6978a336b2',
 'dg.MD1R/e773de58-2137-4c74-ba5f-e35a39cd1940',
 'dg.MD1R/89bef9ef-748f-4844-b4fa-44f8227f3ae8',
 'dg.MD1R/e02f5b2f-3cc0-4af0-bb83-7d6b5aeb8a14',
 'dg.MD1R/f26f445a-6baa-4aad-b2b4-f328485e3ebc',
 'dg.MD1R/bfe72fca-beff-42a4-a420-117b72985e4c',
 'dg.MD1R/b16ddbc9-0580-4b39-8553-43e39bc6e943',
 'dg.MD1R/4904c50b-5c70-4946-9602-f4fc53f70c4f',
 'dg.MD1R/c5acf982-2c9a-4095-9ec6-0f77b9ef202a',
 'dg.MD1R/be8c687d-0e7c-47f7-bcb1-215e58240700',
 'dg.MD1R/4766a010-bb4e-4980-a3fb-cfe79fc951d2',
 'dg.MD1R/4cad42ef-738a-43e1-a267-2c9bf532dfc4',
 'dg.MD1R/2bdf83d3-a1e7-42ed-ac94-75f77dd6fa4c',
 'dg.MD1R/fd766ccd-6079-4a53-8ebf-dff5965f20cc',
 'dg.MD1R/9d554520-eee7-450e-82ce-1c4652306630',
 'dg.MD1R/f9391b0e-4fe1-4203-98a0-a9f4581b928f',
 'dg.MD1R/a7a4c3df-f3c1-4865-ab56-988032ad8467',
 'dg.MD1R/b289a428-9f34-42f1-8ed5-132c8fe7faae',
 'dg.MD1R/192b660f-65cf-4483-9f57-60b4230b0e84',
 'dg.MD1R/bd9ade55-9d55-4544-beaf-ed353d00bc0c',
 'dg.MD1R/e1fbde7f-51f9-45b4-9646-283f4ff42409',
 'dg.MD1R/fe32b4e9-abf1-403b-81c5-a21d61a08eee',
 'dg.MD1R/654eacce-983a-4559-86a3-f73fbaa88573',
 'dg.MD1R/c78a1457-ba19-435d-879d-564bad0449f9',
 'dg.MD1R/2c406120-1f27-4200-9fe2-cd81a27a5e7b',
 'dg.MD1R/024cad6c-3b2f-4174-a8f4-80cd5efcb7a1',
 'dg.MD1R/f459fcbd-eab5-4d4b-a246-707e26da8a8e',
 'dg.MD1R/3ab39687-0703-4469-9341-936a0d32e60b',
 'dg.MD1R/37d29afd-483c-4c52-b0bb-0c8cd24e7606',
 'dg.MD1R/f74f72bb-52a8-413c-b814-fcb16209342c',
 'dg.MD1R/fd80a388-bbfb-4eef-a36a-603f1b606bbf',
 'dg.MD1R/dbfb9307-b932-4e5b-bee8-3dce092a8f92',
 'dg.MD1R/5fd34929-f0a2-4be0-b3b0-1c10796aff6b',
 'dg.MD1R/75551578-2a75-46b6-a2c2-1f011220f2f9',
 'dg.MD1R/cce49c8b-3a56-456b-b862-f7cd6d8273aa',
 'dg.MD1R/17f31089-7aec-4e28-9229-e6562750c467',
 'dg.MD1R/eedd525f-49ba-4d25-8017-1e3340017b4e',
 'dg.MD1R/923550b4-6652-40dd-84fb-ddeb5d8284cd',
 'dg.MD1R/4b300703-e679-4773-8f36-7d8ad1ad499d',
 'dg.MD1R/1e3b5e47-7837-4725-8f27-96467413b961',
 'dg.MD1R/1dc2c7e9-c26d-4a20-b0aa-c080e213e0a9',
 'dg.MD1R/2ee02c06-871e-4929-abd2-4c4200cf216e',
 'dg.MD1R/cd91805a-fcef-483d-9e2a-83ebc3841692',
 'dg.MD1R/c6437345-8ceb-4b72-9235-2d646495867c',
 'dg.MD1R/f4d707cd-689b-4cda-a0f7-166e5d483582',
 'dg.MD1R/a52cc33e-a14f-4537-a8f7-49ee35356cf8',
 'dg.MD1R/5880503b-da48-40c1-a17f-9e04b253bd53',
 'dg.MD1R/649ab7ac-2252-4b2e-9c0d-3094e426a2db',
 'dg.MD1R/94a00a00-e9be-4f82-928b-586810f71514',
 'dg.MD1R/68f72b23-e46b-4606-b278-f15c766c599c',
 'dg.MD1R/265bca45-7e73-435e-949f-bc44f98c8f99',
 'dg.MD1R/7cac2d73-19f2-4726-b0db-2bb737e74e49',
 'dg.MD1R/e3d1cb87-3676-4137-b7db-45cc916716de',
 'dg.MD1R/93afa30d-17ef-4a96-a5f2-c8e0facfcf52',
 'dg.MD1R/e6b7fb68-2762-4cc2-a44d-93c08375c90b',
 'dg.MD1R/94b41821-a4f7-45a4-a58c-5d77fd7cafb1',
 'dg.MD1R/3cffad56-31db-40f8-90a8-16d056b7ae8a',
 'dg.MD1R/c4fc4b0d-e019-483b-a000-f18ce51a9ecc',
 'dg.MD1R/68900751-2c5d-4c5f-b84b-c002063aa0e3',
 'dg.MD1R/4da7d7cd-ec34-4117-a275-85ee5024027f',
 'dg.MD1R/7d5e6160-29c8-49ca-b3b1-55576b7fc8ed',
 'dg.MD1R/9b60dba8-4cae-4dd8-be82-f6b956598253',
 'dg.MD1R/3097bdfd-f706-4698-b287-6faf7ba8aab3',
 'dg.MD1R/10a9c9a2-3adf-430a-aada-3c9cff1b3f65',
 'dg.MD1R/28c74ac9-d6cd-435f-9afb-16748c5b35d1',
 'dg.MD1R/4d3befcd-2373-4965-9e84-6253654abe6f',
 'dg.MD1R/485b56ac-5ae9-4054-a408-87e983ca0b37',
 'dg.MD1R/63e4c3be-fb9d-4114-bb1e-4584d6876355',
 'dg.MD1R/be66c9b4-4fcf-4b0a-af45-abc3327bb0e6',
 'dg.MD1R/3dc8233c-a728-4f8c-8b3b-44042e638764',
 'dg.MD1R/805b94dd-88ed-46f9-ab90-7efe99c61344',
 'dg.MD1R/e61fbaa2-2001-486f-89e6-b5428477af16',
 'dg.MD1R/6c676133-a793-4ae2-ae50-871b80fca327',
 'dg.MD1R/cea04271-975e-4f4f-b190-1cb575aa375e',
 'dg.MD1R/f2d95710-bfdb-49f5-b95d-8589bdcff7fe',
 'dg.MD1R/9b7936f2-24c4-41f9-ad68-d2f7db8068cd',
 'dg.MD1R/ab4b6e4e-885c-4f61-98ee-998d54555e89',
 'dg.MD1R/767b6282-0144-4ba7-9942-e9b78a87776e',
 'dg.MD1R/917b8738-9ffa-4c35-a7cf-c760bc663c52',
 'dg.MD1R/9c61ec3c-34c1-4137-9460-1c9cf96e01e3',
 'dg.MD1R/99c73be7-4adb-46e5-baf8-4293957bc642',
 'dg.MD1R/9b5e226b-f52c-4822-9d42-981887cf302f',
 'dg.MD1R/81e4266f-5143-440c-9872-f5c6ca05e6c3',
 'dg.MD1R/db8dee32-3b6a-4f0e-8210-65b1e7eee25b',
 'dg.MD1R/c1597efc-2867-4eff-97ff-6b24e5d03ef7',
 'dg.MD1R/102ddf9f-dc39-454a-b033-f8825cc2c55d',
 'dg.MD1R/42cd3a03-a608-463c-b650-5ad8f49dc5f3',
 'dg.MD1R/c03305e1-52ee-4de5-83e4-fcfc303c9393',
 'dg.MD1R/02450f17-f58c-4741-a722-e1521456831d',
 'dg.MD1R/b9fba2c0-4689-435f-9f84-fae6a89e824e',
 'dg.MD1R/ca3b3a5d-6fed-41c9-892a-33ceed5a019b',
 'dg.MD1R/361ec8ca-f364-4ccb-9111-cfd3cd07c2fd',
 'dg.MD1R/4e1a3869-9004-420d-a961-56add39c0976',
 'dg.MD1R/f3852f8c-2ac1-4ee9-a573-849dbbd79c82',
 'dg.MD1R/96d2a609-23d8-4921-8575-2172785e51c9',
 'dg.MD1R/4ac78568-4fe3-4e1a-aae0-d309e36c994d',
 'dg.MD1R/119084bf-9d17-421f-9d3e-dae377f0521e',
 'dg.MD1R/2903b01e-82eb-4b55-84b1-021944ba7c73',
 'dg.MD1R/e3801ef4-5284-46f4-936d-4c34a853c464',
 'dg.MD1R/d26bb2a0-941f-4351-a702-d4187e0ad01a',
 'dg.MD1R/43c35bc7-941a-4d83-8a1b-15c4f4d5b0cd',
 'dg.MD1R/a91f5228-c0d6-4248-8214-ad99ebdeed24',
 'dg.MD1R/fe77daf0-6b5e-442b-a170-31562e002dd6',
 'dg.MD1R/fd6d5c17-6742-402f-8fb0-e58323db972e',
 'dg.MD1R/ed93e2d0-2fbd-4394-a354-fa30200e37ff',
 'dg.MD1R/a648e284-0a7e-4075-926b-5c2695aaad28',
 'dg.MD1R/ccba5f2b-d939-444f-b885-2870612ffd38',
 'dg.MD1R/f6cb4789-8b54-42f2-a806-7582ce07d0c2',
 'dg.MD1R/bdc3bfd3-0b11-491f-85b3-38f3462f99eb',
 'dg.MD1R/6d675404-c1bb-485d-9fea-9f606f58e3ef',
 'dg.MD1R/3c346ed7-e57f-4b43-b899-aa852bbca371',
 'dg.MD1R/bcef5aa6-c18b-4e7b-9df6-f3d6198913c9',
 'dg.MD1R/efc57dd2-c07e-4192-988c-d7407f1e659d',
 'dg.MD1R/90a4fa64-2f92-4eab-87e9-a22e2c8a3c71',
 'dg.MD1R/c6e91209-3794-47b3-852b-7788e83b695d',
 'dg.MD1R/c26ac5d3-98c3-48b4-869b-a41fd9078d30',
 'dg.MD1R/217ae54b-4a4c-4342-8763-e0a858333966',
 'dg.MD1R/26589b86-0c6b-4c83-9e58-0484958fed1f',
 'dg.MD1R/ea04155b-ffd9-40cf-bc5a-8c4c7f6db288',
 'dg.MD1R/44dadd04-4a96-407e-890d-29c47057d4fc',
 'dg.MD1R/02d2d359-f4c3-4b35-89da-fcf335adb7cf',
 'dg.MD1R/864296fc-5ec7-419c-afdf-c05e6622b99f',
 'dg.MD1R/7c8d8441-31da-4a8b-a6b8-20bc3edee610',
 'dg.MD1R/2d2d2dff-82cb-425e-8a0f-924c96202322',
 'dg.MD1R/2f6862d7-a8c2-482d-ab20-c4678cd17d42',
 'dg.MD1R/b044739d-39d3-42e7-bfce-c4c48ea58a61',
 'dg.MD1R/d72486ce-3251-4d93-9912-b9444f5e1d62',
 'dg.MD1R/14eebffe-c444-4f02-8781-fc1070e9ab31',
 'dg.MD1R/0d2dcba5-bdcc-4ce5-952d-f3d18eeaecea',
 'dg.MD1R/1ed12552-366e-4e23-9a3f-f8cffcc79dac',
 'dg.MD1R/fc0e2056-8581-4fe2-b51f-6b4955951d14',
 'dg.MD1R/4c1fe673-c5d4-47a7-8ead-f4865957773e',
 'dg.MD1R/9f99629c-ccca-4b6c-ae42-8f4c2564e5a2',
 'dg.MD1R/0f3a3e6f-eae8-48de-ad3f-d159b76d6bf4',
 'dg.MD1R/bf19641c-e4cb-454b-a35c-b722dfe8935a',
 'dg.MD1R/b45cba30-4259-421a-ad7d-51c69b857262',
 'dg.MD1R/08d88a7a-0151-4b41-96c3-2519b0e8590e',
 'dg.MD1R/1da23c6b-293a-4d15-81f1-faff4fe80832',
 'dg.MD1R/2e3458a1-2637-407d-88be-0cb3f2a4a921',
 'dg.MD1R/338ca72d-c200-429d-a6ae-b3c9daacab15',
 'dg.MD1R/4e9e780e-56a5-41f1-b6d2-5eff3ed663bb',
 'dg.MD1R/5b98c64f-befd-45e5-a234-4e4db6b50382',
 'dg.MD1R/be4d2b21-32c3-4da7-afc3-732ec7cd8b8e']

Queries to Postgres using Peregrine graphQL query service¶


  • Peregrine GitHub Docs: https://github.com/uc-cdis/peregrine
  • Peregrine swagger docs: https://petstore.swagger.io/?url=https://raw.githubusercontent.com/uc-cdis/peregrine/master/openapis/swagger.yaml

  • Most structured data (aka "metadata") submitted to a Gen3 system is stored in Postgres tables using the "sheepdog" service. This data must conform to the data commons' data model (https://data.midrc.org/dd), and is queryable via the "peregrine" service, which converts graphQL queries to SQL queries and returns the data requested. The Postgres tables are considered the "source-of-truth" for data in a Gen3 system (vs. the derived data in ElasticSearch, covered below).

  • On the data commons' website, peregrine queries can be sent to the API using the "graphiQL" query builder: https://data.midrc.org/query (click on "Switch to Graph Model"; if button says "Switch to Flat Model" you're in the correct spot).

  • Alternatively, you can send queries to the peregrine API using the Gen3SDK Gen3Submission.query() function, which uses the Python requests package to send queries as API requests: https://github.com/uc-cdis/gen3sdk-python/blob/31751633ba621b35f39eda7295f131245fb92728/gen3/submission.py#L399

Example graph model query #1¶

  • This query is running across all records in the case node and returns data from any dataset in the data commons you are authorized to access. Remember, the properties in the case node are essentially table headers for variables whose values are of a specific data type (string, enumeration, integer, number, boolean, array, etc.).
  • The argument covid19_positive: "Yes" returns only case records that have the value "Yes" for the property covid19_positive, which indicates whether a case in MIDRC has ever had a positive COVID-19 test result.
  • The first argument defines how many case records we want returned. Using the argument first: 0, all the records we have access to will be returned. If we leave the "first" argument out, only the first 10 records are returned by default. Setting first: 2000 will return the first 2000 records in the table, etc.
  • If your query is timing out, you will need to paginate the query (covered in next section) using a combination of "first" and "offset" arguments. This is only necessary if the tables being queries are very large, or the query traverses many nodes in the graph model.
  • Properties we want returned from the API are enclosed in brackets. The possibilities and exact syntax are constrained by the data model (data.midrc.org/dd).
In [55]:
### Define the query

## Here we're asking for the `project_id`, `submitter_id`, and some demographic data for every `case` record.
## We're also asking for the `study_uid` for every `imaging_study` record belonging to those cases, and for all `dx_series_file` records for those `imaging_studies`.
## Finally, we're asking for the `file_name` and `object_id` of any Digital X-ray files (node `dx_series_file`, backref: `dx_series_files`) they may have.

## Note: "submitter_id" is a required property on every node, which is the human-readable (string), unique identifier for a record in a data table / node. So, the "submitter_id" of a record in the case node is the de-identified patient's "ID".

query_txt = """
{
  case(first: 0, covid19_positive: "Yes") {
    project_id
    submitter_id
    ethnicity
    sex
    race
    imaging_studies (study_modality: "DX") {
      study_uid
      dx_series_files {
        object_id
        file_name
      }
    }
  }
}

"""
In [56]:
## Send the query using the Gen3 SDK Gen3Submission.query() function
## The response will be in JSON format.

response = sub.query(query_txt)
if 'data' in response:
    data = response['data']['case']
    display(data)
[{'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [{'dx_series_files': [{'file_name': '2.16.840.1.114274.1818.46462464228338035604085382488501697711/2.16.840.1.114274.1818.57119669613829751891125487061413366952.zip',
      'object_id': 'dg.MD1R/5eeb7280-7d5b-4bac-a34b-a0eadb65e033'}],
    'study_uid': '2.16.840.1.114274.1818.46462464228338035604085382488501697711'}],
  'project_id': 'Open-A1',
  'race': 'Black or African American',
  'sex': 'Female',
  'submitter_id': '10000364-1163342'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Female',
  'submitter_id': '10000364-1560988'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [{'dx_series_files': [{'file_name': '2.16.840.1.114274.1818.462269544687285955216049365983114851242/2.16.840.1.114274.1818.5266294379737328289107155289649633448.zip',
      'object_id': 'dg.MD1R/1e1a8278-4fb8-46cc-8ca4-53e1931de592'}],
    'study_uid': '2.16.840.1.114274.1818.462269544687285955216049365983114851242'},
   {'dx_series_files': [{'file_name': '10000364-1605860/2.16.840.1.114274.1818.49175781962090677039777902775594799259/2.16.840.1.114274.1818.51284993071612923214127329616632366210.zip',
      'object_id': 'dg.MD1R/5b06fbfb-a92d-466f-9d93-6b3fab850e5c'}],
    'study_uid': '2.16.840.1.114274.1818.49175781962090677039777902775594799259'}],
  'project_id': 'Open-A1',
  'race': 'Black or African American',
  'sex': 'Female',
  'submitter_id': '10000364-1605860'},
 {'ethnicity': 'Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'Not Reported',
  'sex': 'Female',
  'submitter_id': '10000364-6388914'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [{'dx_series_files': [{'file_name': '10000364-1958844/2.16.840.1.114274.1818.52236113359126249589212595743121753735/2.16.840.1.114274.1818.54309100269617797736626917868992258958.zip',
      'object_id': 'dg.MD1R/ea6ad8e7-1dc9-4916-8e75-38abb66c6416'}],
    'study_uid': '2.16.840.1.114274.1818.52236113359126249589212595743121753735'}],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Female',
  'submitter_id': '10000364-1958844'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Female',
  'submitter_id': '10000364-2421812'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'Asian',
  'sex': 'Female',
  'submitter_id': '10000364-769690'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Female',
  'submitter_id': '10000364-1034580'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Female',
  'submitter_id': '10000364-1162208'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Female',
  'submitter_id': '10000364-1184584'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Female',
  'submitter_id': '10000364-120722'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'Black or African American',
  'sex': 'Female',
  'submitter_id': '10000364-1242909'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [{'dx_series_files': [{'file_name': '10000364-1364048/2.16.840.1.114274.1818.461965818635501599112499939473304930980/2.16.840.1.114274.1818.56164105254341726913153672955035750539.zip',
      'object_id': 'dg.MD1R/a0c8e617-3430-45d0-b1e8-da193161f6f3'},
     {'file_name': '10000364-1364048/2.16.840.1.114274.1818.461965818635501599112499939473304930980/2.16.840.1.114274.1818.48131375082244828448653030814453009285.zip',
      'object_id': 'dg.MD1R/68c0d3bf-06d9-4561-9f38-926c3948902a'}],
    'study_uid': '2.16.840.1.114274.1818.461965818635501599112499939473304930980'}],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Male',
  'submitter_id': '10000364-1364048'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'White',
  'sex': 'Female',
  'submitter_id': '10000364-140041'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'Black or African American',
  'sex': 'Female',
  'submitter_id': '10000364-1409534'},
 {'ethnicity': 'Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'Not Reported',
  'sex': 'Female',
  'submitter_id': '10000364-1463758'},
 {'ethnicity': 'Not Hispanic or Latino',
  'imaging_studies': [],
  'project_id': 'Open-A1',
  'race': 'Asian',
  'sex': 'Male',
  'submitter_id': '10000364-1508090'},
 ...]
In [57]:
## the "object_id" field is the file's data GUID (or globally unique identifier), which can be used to access the file.

object_ids = []
for case in data:
    studies = case['imaging_studies']
    for study in studies:
        files = study['dx_series_files']
        if len(files)>0:
            for file in files:
                object_id = file['object_id']
                object_ids.append(object_id)
object_ids
Out[57]:
['dg.MD1R/5eeb7280-7d5b-4bac-a34b-a0eadb65e033',
 'dg.MD1R/1e1a8278-4fb8-46cc-8ca4-53e1931de592',
 'dg.MD1R/5b06fbfb-a92d-466f-9d93-6b3fab850e5c',
 'dg.MD1R/ea6ad8e7-1dc9-4916-8e75-38abb66c6416',
 'dg.MD1R/a0c8e617-3430-45d0-b1e8-da193161f6f3',
 'dg.MD1R/68c0d3bf-06d9-4561-9f38-926c3948902a',
 'dg.MD1R/06f0d3ab-113b-45b2-84a4-8ba9242af0b6',
 'dg.MD1R/f1242600-47d1-4ce0-9851-8c0a976a0e29',
 'dg.MD1R/27acfaeb-bf00-4cbb-a5cc-4b74ed3c9c1d',
 'dg.MD1R/9bd41c93-7e4a-4267-84ad-51970dc802c1',
 'dg.MD1R/131b058a-947a-482b-b3ab-3afc7a179540',
 'dg.MD1R/5c487296-09ff-4339-90e5-3c23bac0f973',
 'dg.MD1R/43c36d1b-2975-4b0d-953f-0e9e71a65191',
 'dg.MD1R/ac842239-0e5d-400f-a59d-05ad53ce09b2',
 'dg.MD1R/e74025f3-758a-4c1e-a07f-d9b59d796436',
 'dg.MD1R/b333cc66-70d9-43e5-9167-6b9d85464388',
 'dg.MD1R/8af38ef3-bc22-49c0-85d8-618ace26428f',
 'dg.MD1R/360db42d-c1a4-4be6-adda-b17da1527d86',
 'dg.MD1R/5732e1fb-628f-4b92-be65-b8e59dafef5f',
 'dg.MD1R/1c83429e-9508-4b76-9a3a-f83bfa7c31ea',
 'dg.MD1R/57a8f3c5-b914-4225-a2dd-2c26e655ba08',
 'dg.MD1R/ddba0a51-d2cc-4fd6-a1e5-5f807e5b7860',
 'dg.MD1R/4ba5a894-3126-464b-adb9-919472df9128',
 'dg.MD1R/aff01648-db31-4241-bdd6-40eda6729091',
 'dg.MD1R/e998c3f3-2a76-4a6c-95f3-e623c4b8d96a',
 'dg.MD1R/d2352822-a475-4cbe-a96a-1decc2fad41c',
 'dg.MD1R/94b290ba-9340-4f5c-a3a3-561ceb4d7379',
 'dg.MD1R/3f67e943-0c3c-4cc1-8183-b43ada1c9a6d',
 'dg.MD1R/62cf4244-1a9a-4c0f-aa9f-dbd671457e80',
 'dg.MD1R/893a8765-0d5c-457d-b822-b76808d5aab9',
 'dg.MD1R/5673906c-d484-4736-be41-f0db768d75cf',
 'dg.MD1R/e2d8cb19-20fb-4f9e-b85c-ac9c4b1e2f3b',
 'dg.MD1R/460f0996-aa36-48e0-955c-2c31d845b6c1',
 'dg.MD1R/c357eb4c-53dd-46cb-88c1-2a3cbde411d6',
 'dg.MD1R/dc20af3b-c8bd-4f18-8069-4b1e20050a83',
 'dg.MD1R/33c673ab-756a-44d8-baa4-de93885aab48',
 'dg.MD1R/0c370ca5-8eac-4b00-9b3b-d18eb9ef9c7d',
 'dg.MD1R/50518dbf-ef66-4295-8d86-7c18a72be7fc',
 'dg.MD1R/b4f2e513-c29d-4f75-8456-1a26e1882636',
 'dg.MD1R/21259aaa-4462-4f9a-9b43-a8b49afc86b0',
 'dg.MD1R/472a543a-6925-4173-b584-23081dc1d1b3',
 'dg.MD1R/b703817f-34e9-4230-861a-06785f3115dd',
 'dg.MD1R/28357697-68b4-4f0e-aa3a-09798bb3b90e',
 'dg.MD1R/ef43753b-9b2d-455a-87ca-132d2cfaaeda',
 ...]
In [58]:
## Take a look at one of the file objects
object_id = object_ids[0]
os.system("gen3 drs-pull object {}".format(object_id))
Out[58]:
0
In [62]:
!unzip 10000364-1163342/2.16.840.1.114274.1818.46462464228338035604085382488501697711/2.16.840.1.114274.1818.57119669613829751891125487061413366952.zip
Archive:  10000364-1163342/2.16.840.1.114274.1818.46462464228338035604085382488501697711/2.16.840.1.114274.1818.57119669613829751891125487061413366952.zip
 extracting: 2.16.840.1.114274.1818.57119669613829751891125487061413366952/2.16.840.1.114274.1818.48835068669285345335314100396583670180.dcm
 extracting: 2.16.840.1.114274.1818.57119669613829751891125487061413366952/2.16.840.1.114274.1818.50562724593808174072277459636544006786.dcm
 extracting: 2.16.840.1.114274.1818.57119669613829751891125487061413366952/2.16.840.1.114274.1818.545101927693002822610905640249661878717.dcm
 extracting: 2.16.840.1.114274.1818.57119669613829751891125487061413366952/2.16.840.1.114274.1818.572332279246432020710965140097547833765.dcm
In [66]:
from pydicom import dcmread

fpath = "2.16.840.1.114274.1818.57119669613829751891125487061413366952/2.16.840.1.114274.1818.48835068669285345335314100396583670180.dcm"
ds = dcmread(fpath)
ds
Out[66]:
Dataset.file_meta -------------------------------
(0002, 0000) File Meta Information Group Length  UL: 220
(0002, 0001) File Meta Information Version       OB: b'\x00\x01'
(0002, 0002) Media Storage SOP Class UID         UI: Digital X-Ray Image Storage - For Presentation
(0002, 0003) Media Storage SOP Instance UID      UI: 2.16.840.1.114274.1818.48835068669285345335314100396583670180
(0002, 0010) Transfer Syntax UID                 UI: JPEG Lossless, Non-Hierarchical, First-Order Prediction (Process 14 [Selection Value 1])
(0002, 0012) Implementation Class UID            UI: 1.3.6.1.4.1.25403.1.1.1
(0002, 0013) Implementation Version Name         SH: 'Dicom 0.1'
(0002, 0016) Source Application Entity Title     AE: 'CPV-ACUOAPP1'
-------------------------------------------------
(0008, 0005) Specific Character Set              CS: ''
(0008, 0008) Image Type                          CS: ['ORIGINAL', 'PRIMARY', '']
(0008, 0012) Instance Creation Date              DA: ''
(0008, 0013) Instance Creation Time              TM: '000000'
(0008, 0016) SOP Class UID                       UI: Digital X-Ray Image Storage - For Presentation
(0008, 0018) SOP Instance UID                    UI: 2.16.840.1.114274.1818.48835068669285345335314100396583670180
(0008, 0020) Study Date                          DA: '19600225'
(0008, 0021) Series Date                         DA: '19600225'
(0008, 0022) Acquisition Date                    DA: '19600225'
(0008, 0023) Content Date                        DA: '19600225'
(0008, 002a) Acquisition DateTime                DT: '19600706172810.000000'
(0008, 0030) Study Time                          TM: '172612.000'
(0008, 0031) Series Time                         TM: '172613.000'
(0008, 0032) Acquisition Time                    TM: '172813'
(0008, 0033) Content Time                        TM: '172813.000'
(0008, 0050) Accession Number                    SH: ''
(0008, 0060) Modality                            CS: 'DX'
(0008, 0061) Modalities in Study                 CS: ['', 'DX', '']
(0008, 0068) Presentation Intent Type            CS: 'FOR PRESENTATION'
(0008, 0070) Manufacturer                        LO: 'GE Healthcare'
(0008, 0090) Referring Physician's Name          PN: ''
(0008, 1010) Station Name                        SH: '[Statio.825]'
(0008, 1030) Study Description                   LO: 'XR CHEST PA AND LATERAL'
(0008, 103e) Series Description                  LO: 'Chest'
(0008, 1090) Manufacturer's Model Name           LO: 'Discovery XR656 HD'
(0008, 1110)  Referenced Study Sequence  0 item(s) ----
(0008, 2112)  Source Image Sequence  1 item(s) ----
   (0008, 1150) Referenced SOP Class UID            UI: Digital X-Ray Image Storage - For Processing
   (0008, 1155) Referenced SOP Instance UID         UI: 2.16.840.1.114274.1818.522064827453477483917962389381240942983
   ---------
(0008, 2218)  Anatomic Region Sequence  1 item(s) ----
   (0008, 0100) Code Value                          SH: 'T-D3000'
   (0008, 0102) Coding Scheme Designator            SH: 'SRT'
   (0008, 0104) Code Meaning                        LO: 'Chest'
   ---------
(0008, 3010) Irradiation Event UID               UI: 2.16.840.1.114274.1818.51573432723524135609081252859215753367
(0010, 0010) Patient's Name                      PN: '1163342^10000364^^^'
(0010, 0020) Patient ID                          LO: '10000364-1163342'
(0010, 0030) Patient's Birth Date                DA: ''
(0010, 1010) Patient's Age                       AS: '049Y'
(0010, 1020) Patient's Size                      DS: '1.727'
(0010, 1030) Patient's Weight                    DS: '126.009'
(0012, 0010) Clinical Trial Sponsor Name         LO: '[Clinical Trial Sponsor Na]'
(0012, 0020) Clinical Trial Protocol ID          LO: '[Clinical Trial Protocol ]'
(0012, 0021) Clinical Trial Protocol Name        LO: 'CIRR'
(0012, 0030) Clinical Trial Site ID              LO: '[Clinical Trial Site ]'
(0012, 0031) Clinical Trial Site Name            LO: '[Clinical Trial Site Na]'
(0012, 0040) Clinical Trial Subject ID           LO: '10000364-1163342'
(0012, 0050) Clinical Trial Time Point ID        LO: '[Clinical Trial Time Point ]'
(0012, 0060) Clinical Trial Coordinating Center  LO: '[Clinical Trial Coordinating Center Na]'
(0012, 0062) Patient Identity Removed            CS: 'YES'
(0012, 0063) De-identification Method            LO: ['Basic Application Confidentiality Profile', 'Retain Longitudinal Temporal Information Modified Dates', 'TRIAD Anonymization; Date=20210824; Time=134315']
(0012, 0064)  De-identification Method Code Sequence  3 item(s) ----
   (0008, 0100) Code Value                          SH: '113100'
   (0008, 0102) Coding Scheme Designator            SH: 'DCM'
   (0008, 0104) Code Meaning                        LO: 'Basic Application Confidentiality Profile'
   ---------
   (0008, 0100) Code Value                          SH: '113107'
   (0008, 0102) Coding Scheme Designator            SH: 'DCM'
   (0008, 0104) Code Meaning                        LO: 'Retain Longitudinal Temporal Information Modified Dates Option'
   ---------
   (0008, 0100) Code Value                          SH: '900901'
   (0008, 0102) Coding Scheme Designator            SH: 'WIF'
   (0008, 0104) Code Meaning                        LO: 'BASIC PROFILE OVERRIDES POTENTIALLY VIOLATE SECURITY STANDARD'
   (0008, 0108) Extended Code Meaning               LT: Array of 229 elements
   ---------
(0018, 0010) Contrast/Bolus Agent                LO: ''
(0018, 0015) Body Part Examined                  CS: 'CHEST'
(0018, 0060) KVP                                 DS: '120.0'
(0018, 1010) Secondary Capture Device ID         LO: ''
(0018, 1020) Software Versions                   LO: 'DM_PLATFORM_XRAY_DUCKWEED_4.8.3_2020_FW41.2_65'
(0018, 1030) Protocol Name                       LO: '[Protocol Na]'
(0018, 1110) Distance Source to Detector         DS: '1828.0'
(0018, 1111) Distance Source to Patient          DS: '1790.0'
(0018, 1147) Field of View Shape                 CS: 'RECTANGLE'
(0018, 1149) Field of View Dimension(s)          IS: [426, 350]
(0018, 1150) Exposure Time                       IS: '47'
(0018, 1151) X-Ray Tube Current                  IS: '394'
(0018, 1152) Exposure                            IS: '18'
(0018, 1153) Exposure in uAs                     IS: '18323'
(0018, 115e) Image and Fluoroscopy Area Dose Pro DS: '8.771052'
(0018, 1160) Filter Type                         SH: 'NONE'
(0018, 1164) Imager Pixel Spacing                DS: [0.1, 0.1]
(0018, 1166) Grid                                CS: ['FIXED', 'FOCUSED']
(0018, 1190) Focal Spot(s)                       DS: '1.3'
(0018, 1405) Relative X-Ray Exposure             IS: '821'
(0018, 1411) Exposure Index                      DS: '184.12'
(0018, 1412) Target Exposure Index               DS: '164.810993'
(0018, 1413) Deviation Index                     DS: '0.5'
(0018, 1508) Positioner Type                     CS: ''
(0018, 1700) Collimator Shape                    CS: 'POLYGONAL'
(0018, 1720) Vertices of the Polygonal Collimato IS: [4259, 3495, 0, 3495, 0, 0, 4259, 0]
(0018, 5101) View Position                       CS: 'LL'
(0018, 6000) Sensitivity                         DS: '596.647186'
(0018, 7000) Detector Conditions Nominal Flag    CS: 'YES'
(0018, 7001) Detector Temperature                DS: '32.507999'
(0018, 7004) Detector Type                       CS: 'SCINTILLATOR'
(0018, 7005) Detector Configuration              CS: 'AREA'
(0018, 701a) Detector Binning                    DS: [1, 1]
(0018, 7020) Detector Element Physical Size      DS: [0.100000, 0.100000]
(0018, 7022) Detector Element Spacing            DS: [0.100000, 0.100000]
(0018, 7024) Detector Active Shape               CS: 'RECTANGLE'
(0018, 7026) Detector Active Dimension(s)        DS: [426.000000, 349.600000]
(0018, 7030) Field of View Origin                DS: [0, 0]
(0018, 7032) Field of View Rotation              DS: '0.0'
(0018, 7034) Field of View Horizontal Flip       CS: 'NO'
(0018, 704c) Grid Focal Distance                 DS: '1300.0'
(0018, 7050) Filter Material                     CS: 'COPPER'
(0018, 7052) Filter Thickness Minimum            DS: '0.0'
(0018, 7054) Filter Thickness Maximum            DS: '0.0'
(0018, 7060) Exposure Control Mode               CS: 'AUTOMATIC'
(0018, 7062) Exposure Control Mode Description   LT: Array of 24 elements
(0018, a001)  Contributing Equipment Sequence  1 item(s) ----
   (0008, 0070) Manufacturer                        LO: 'Claron'
   (0008, 1010) Station Name                        SH: 'CPV-ACUOPRIAPP'
   (0008, 1090) Manufacturer's Model Name           LO: 'Deidentification'
   (0018, 1020) Software Versions                   LO: ['WIF 2.1.17', '1.0.0.0']
   (0018, a002) Contribution DateTime               DT: '20211106054117.245042'
   (0018, a003) Contribution Description            ST: 'Cleaned'
   (0040, a170)  Purpose of Reference Code Sequence  1 item(s) ----
      (0008, 0100) Code Value                          SH: '109104'
      (0008, 0102) Coding Scheme Designator            SH: 'DCM'
      (0008, 0104) Code Meaning                        LO: 'De-identifying Equipment'
      ---------
   ---------
(0020, 000d) Study Instance UID                  UI: 2.16.840.1.114274.1818.46462464228338035604085382488501697711
(0020, 000e) Series Instance UID                 UI: 2.16.840.1.114274.1818.57119669613829751891125487061413366952
(0020, 0010) Study ID                            SH: ''
(0020, 0011) Series Number                       IS: '1780'
(0020, 0013) Instance Number                     IS: '1'
(0020, 0022) Overlay Number                      IS: None
(0020, 0024) Curve Number                        IS: None
(0020, 0026) LUT Number                          IS: None
(0020, 0062) Image Laterality                    CS: 'U'
(0028, 0002) Samples per Pixel                   US: 1
(0028, 0004) Photometric Interpretation          CS: 'MONOCHROME2'
(0028, 0008) Number of Frames                    IS: '1'
(0028, 0010) Rows                                US: 4259
(0028, 0011) Columns                             US: 3495
(0028, 0030) Pixel Spacing                       DS: [0.097921, 0.097921]
(0028, 0100) Bits Allocated                      US: 16
(0028, 0101) Bits Stored                         US: 16
(0028, 0102) High Bit                            US: 15
(0028, 0103) Pixel Representation                US: 0
(0028, 0300) Quality Control Image               CS: 'NO'
(0028, 0301) Burned In Annotation                CS: 'NO'
(0028, 0303) Longitudinal Temporal Information M CS: 'MODIFIED'
(0028, 0a02) Pixel Spacing Calibration Type      CS: 'GEOMETRY'
(0028, 0a04) Pixel Spacing Calibration Descripti LO: 'Corrected by the SOD and SID'
(0028, 1040) Pixel Intensity Relationship        CS: 'LOG'
(0028, 1041) Pixel Intensity Relationship Sign   SS: -1
(0028, 1050) Window Center                       DS: [18447, 18447, 18447]
(0028, 1051) Window Width                        DS: [18810, 14107, 28215]
(0028, 1052) Rescale Intercept                   DS: '0.0'
(0028, 1053) Rescale Slope                       DS: '1.0'
(0028, 1054) Rescale Type                        LO: 'US'
(0028, 1055) Window Center & Width Explanation   LO: ['NORMAL', 'HARDER', 'SOFTER']
(0028, 2110) Lossy Image Compression             CS: '00'
(0032, 000a) Study Status ID                     CS: 'COMPLETED'
(0040, 0009) Scheduled Procedure Step ID         SH: ''
(0040, 0244) Performed Procedure Step Start Date DA: ''
(0040, 0245) Performed Procedure Step Start Time TM: '000000'
(0040, 0260)  Performed Protocol Code Sequence  4 item(s) ----
   (0008, 0100) Code Value                          SH: 'DICXRCOMP'
   (0008, 0102) Coding Scheme Designator            SH: 'L'
   (0008, 0103) Coding Scheme Version               SH: ''
   (0008, 0104) Code Meaning                        LO: 'XR CHEST PA AND LATERAL'
   ---------
   (0008, 0100) Code Value                          SH: 'DICXRCOMP'
   (0008, 0102) Coding Scheme Designator            SH: 'L'
   (0008, 0103) Coding Scheme Version               SH: ''
   (0008, 0104) Code Meaning                        LO: 'XR CHEST PA AND LATERAL'
   ---------
   (0008, 0100) Code Value                          SH: 'DICXRCOMP'
   (0008, 0102) Coding Scheme Designator            SH: 'L'
   (0008, 0103) Coding Scheme Version               SH: ''
   (0008, 0104) Code Meaning                        LO: 'XR CHEST PA AND LATERAL'
   ---------
   (0008, 0100) Code Value                          SH: 'DICXRCOMP'
   (0008, 0102) Coding Scheme Designator            SH: 'L'
   (0008, 0103) Coding Scheme Version               SH: ''
   (0008, 0104) Code Meaning                        LO: 'XR CHEST PA AND LATERAL'
   ---------
(0040, 0302) Entrance Dose                       US: 0
(0040, 0310) Comments on Radiation Dose          ST: '125%'
(0040, 8302) Entrance Dose in mGy                DS: '0.821157'
(0040, a032) Observation DateTime                DT: '18390126000000.000000'
(0040, a491) Completion Flag                     CS: ''
(0040, a493) Verification Flag                   CS: ''
(0045, 0010) Private Creator                     LO: 'GEMS_SEND_03'
(0054, 0220)  View Code Sequence  1 item(s) ----
   (0008, 0100) Code Value                          SH: 'R-10236'
   (0008, 0102) Coding Scheme Designator            SH: 'SRT'
   (0008, 0104) Code Meaning                        LO: 'left lateral'
   ---------
(0054, 0410)  Patient Orientation Code Sequence  1 item(s) ----
   (0008, 0100) Code Value                          SH: 'F-10450'
   (0008, 0102) Coding Scheme Designator            SH: 'SRT'
   (0008, 0104) Code Meaning                        LO: 'recumbent'
   ---------
(0054, 0414)  Patient Gantry Relationship Code Sequence  1 item(s) ----
   (0008, 0100) Code Value                          SH: 'F-10470'
   (0008, 0102) Coding Scheme Designator            SH: 'SRT'
   (0008, 0104) Code Meaning                        LO: 'head-first'
   ---------
(0070, 0080) Content Label                       CS: ''
(0070, 0081) Content Description                 LO: ''
(0070, 0084) Content Creator's Name              PN: ''
(0400, 0500)  Encrypted Attributes Sequence  1 item(s) ----
   (0033, 1215) Private tag data                    OB: b'\xb8\xaf\xc7\x01\xa5>\xd0\xec'
   (0400, 0510) Encrypted Content Transfer Syntax U UI: JPEG Lossless, Non-Hierarchical, First-Order Prediction (Process 14 [Selection Value 1])
   (0400, 0520) Encrypted Content                   OB: Array of 720 elements
   ---------
(2050, 0020) Presentation LUT Shape              CS: 'IDENTITY'
(4008, 0040) Results ID                          SH: ''
(7fe0, 0010) Pixel Data                          OB: Array of 16518594 elements

Counts with peregrine¶


  • Peregrine is able to provide counts of records in nodes. A simple example is to quickly get the count of the numbers of cases and imaging studies in the data commons.
  • You can also add arguments to the counts to, for example, get the number of cases in a particular project or get the imaging studies of a particular modality.
In [67]:
query_txt = "{_case_count}"
print(sub.query(query_txt))
query_txt = "{_imaging_study_count}"
print(sub.query(query_txt))
query_txt = '{CT_studies: _imaging_study_count(study_modality:"CT")}'
print(sub.query(query_txt))
query_txt = '{Open_A1_cases: _case_count(project_id:"Open-A1")}'
print(sub.query(query_txt))
{'data': {'_case_count': 11899}}
{'data': {'_imaging_study_count': 31819}}
{'data': {'CT_studies': 6776}}
{'data': {'Open_A1_cases': 7534}}

Queries of "datanode" using peregrine¶


Another handy trick with peregrine queries is the "datanode" query. "Datanode" isn't a real node in the data model, but is useful way to query all nodes that store file information. For example, if you have a patient ID, you can get all the files associated with that case.

In [ ]:
query_txt = """
{
  datanode(first: 0, case_ids: "10000364-1163342") {
    object_id
    file_name
    modality
  }
}

"""
response = sub.query(query_txt)
if 'data' in response:
    display(response['data']['datanode'])

Queries to ElasticSearch using Guppy graphQL query service¶


  • Guppy Documentation: https://github.com/uc-cdis/guppy/blob/master/doc/queries.md#filters
  • Guppy Download instructions: https://github.com/uc-cdis/guppy/blob/master/doc/download.md
  • ETL (Tube) Documentation: https://github.com/uc-cdis/tube#gen3-etl---a-process-from-postgresql-to-es

  • The Gen3 platform includes services for running an ETL process (Extract, Transform, Load), which is done by the Gen3 ETL service "tube", on the data in Postgres to create flattened tables of the same data in ElasticSearch (ES) for rapid querying performed by the Gen3 query service "guppy".
  • Guppy runs graphql-like queries against the ES database, and can rapidly return derived data like histograms, statistics, aggregations, counts, etc. The tube service uses Spark to create these new tables of data in ES via an ETL mapping, which defines the structure of the new tables and is based on the data model.
  • Since the structure of the data changes via the ETL process, peregrine queries to Postgres will not run using guppy. To explore what is possible to query, use the graphiQL interface / documentation.
  • The "Exploration" app aka "Data Explorer" (data.midrc.org/explore), which uses faceted search to filter the flat data tables in ES, runs off of guppy queries.

Screen%20Shot%202022-06-22%20at%2012.56.01%20PM.png

In [196]:
## an example guppy query, which hits the ElasticSearch database

## define some parameters
pid = 'Open-R1'
node = 'imaging_study'
fields = ["study_uid",
    "study_description",
    "case_ids",
    "object_id"]
filters = {"project_id": pid,
    "covid19_positive" : "Yes",
    "body_part_examined" : "CHEST",
    "study_modality" : "DX"}

## send the guppy query with the SDK class Gen3Query
## Note the "first: 100000", which makes sure we don't just get the default first 10 records
response = query.query(
                data_type=node,
                first=100000,
                fields=fields,
                filters=filters,
                sort_object={"submitter_id": "asc"},
)

# display the returned data
if 'data' in response:
    study_data = response['data'][node]
    display(study_data)
[{'case_ids': ['419639-008913'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.233954434199549545416819083344',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/d7b7eebb-da02-445c-998b-a4dfa41d68c2']},
 {'case_ids': ['419639-011445'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.165257741226593093304120397175',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/36af9693-7026-4995-b234-0c0f2cfd3f72']},
 {'case_ids': ['419639-010353'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.238890717554558220188416385332',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/a3bf32c6-2367-4353-8e78-8f61c89b60b6']},
 {'case_ids': ['419639-008392'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.146322355844825400343463634521',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/ef6aa49b-5d0b-40c5-9c41-223f33763ec8']},
 {'case_ids': ['419639-010353'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.250028057444887461519982701946',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/99351386-a8e5-4e87-98ca-0aaa4dacbfd0']},
 {'case_ids': ['419639-010019'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.214209436277337710302867220026',
  'study_description': 'XR CHEST 1 VIEW AP'},
 {'case_ids': ['419639-010353'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.288831991642481501201971600659',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/97f77b10-d034-4a0e-9e2b-a5f2c73a1f03']},
 {'case_ids': ['419639-009522'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.326603550249896351821065609780',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/0370fc32-0d1d-4740-b200-015771c5a762',
   'dg.MD1R/dc69bcef-f185-45ac-9945-93c9945c3c8a']},
 {'case_ids': ['419639-010904'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.156328865492292182988593607504',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/9047b454-f0df-44d6-a2b9-4b67ad8354fc']},
 {'case_ids': ['419639-011805'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.322674477111141177363190807522',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/dc53856f-cba9-4fc3-8972-b678de5ffe5b',
   'dg.MD1R/ab36a63e-bae0-4f6f-81e2-d694cd8ed81a']},
 {'case_ids': ['419639-010753'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.160265009864701999688647615215',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/8e23f305-c3d1-48b9-9e27-e8c9e29ef91e']},
 {'case_ids': ['419639-011284'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.979540923575692211655123361304',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/73061277-70ce-4bf6-8f39-f6789af33998']},
 {'case_ids': ['419639-010904'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.436794751370697250268756784153',
  'study_description': 'XR CHEST 1 VIEW AP',
  'object_id': ['dg.MD1R/0bd32540-b451-446e-bb44-9d69360ca5b9']},
 ...]
In [197]:
## another example guppy query, which hits the ElasticSearch database

## define some parameters
node = 'case'

fields = ["project_id",
    "submitter_id",
    "object_id"]

filters = {"sex":"Female",
    "race" : "Asian",
    "ethnicity" : "Hispanic or Latino"}

## send the guppy query with the SDK class Gen3Query
## Note the "first: 100000", which makes sure we don't just get the default first 10 records
response = query.query(
                data_type=node,
                first=100000,
                fields=fields,
                filters=filters,
                sort_object={"submitter_id": "asc"},
)

# display the returned data
if 'data' in response:
    case_data = response['data'][node]
    display(case_data)
[{'project_id': 'Open-A1',
  'submitter_id': '10000364-1599931',
  'object_id': ['dg.MD1R/b5cebe0d-907a-4ab2-aef7-b96cebd7d448']},
 {'project_id': 'Open-A1',
  'submitter_id': '10008204-ONymhwnpKUy5dlM6b4a5Ng',
  'object_id': ['dg.MD1R/1355977e-81ed-4ac5-934d-b20407eab280',
   'dg.MD1R/ed0954b6-f76d-4e74-bb7e-3818d69d8001',
   'dg.MD1R/109daa01-5969-4c2c-bb44-25efe88856e6',
   'dg.MD1R/6f330232-cee6-4fef-8d11-e5d04a5951f0',
   'dg.MD1R/c32a89c9-eca8-426a-a6b3-a22b7700ecbc',
   'dg.MD1R/9e1a772a-f5b8-485d-9f30-600e1c052313',
   'dg.MD1R/8ca85564-7d98-4312-8507-64f173333da9']}]
In [198]:
## Elastic search is handy for accessing files for a cohort since object_ids associated with each study or case are joined to the table 
study_object_ids = []
for study in study_data:
    if 'object_id' in study:
        object_id_list = study['object_id']
        for object_id in object_id_list:
            study_object_ids.append(object_id)
display(study_object_ids)
['dg.MD1R/d7b7eebb-da02-445c-998b-a4dfa41d68c2',
 'dg.MD1R/36af9693-7026-4995-b234-0c0f2cfd3f72',
 'dg.MD1R/a3bf32c6-2367-4353-8e78-8f61c89b60b6',
 'dg.MD1R/ef6aa49b-5d0b-40c5-9c41-223f33763ec8',
 'dg.MD1R/99351386-a8e5-4e87-98ca-0aaa4dacbfd0',
 'dg.MD1R/97f77b10-d034-4a0e-9e2b-a5f2c73a1f03',
 'dg.MD1R/0370fc32-0d1d-4740-b200-015771c5a762',
 'dg.MD1R/dc69bcef-f185-45ac-9945-93c9945c3c8a',
 'dg.MD1R/9047b454-f0df-44d6-a2b9-4b67ad8354fc',
 'dg.MD1R/dc53856f-cba9-4fc3-8972-b678de5ffe5b',
 'dg.MD1R/ab36a63e-bae0-4f6f-81e2-d694cd8ed81a',
 'dg.MD1R/8e23f305-c3d1-48b9-9e27-e8c9e29ef91e',
 'dg.MD1R/73061277-70ce-4bf6-8f39-f6789af33998',
 'dg.MD1R/0bd32540-b451-446e-bb44-9d69360ca5b9',
 'dg.MD1R/d37ca3d8-42e4-4dd2-94e5-288c9daea1a3',
 'dg.MD1R/a54c1a34-af22-4c3b-a30b-b67ae3ad2e3c',
 'dg.MD1R/879381ef-8697-4fa8-aa4e-6d5d3da8622e',
 'dg.MD1R/a0e6787d-d093-4d03-870b-7ce2cd26aa7f',
 'dg.MD1R/7151bb86-16c9-4baf-ab50-1bebaf663a92',
 'dg.MD1R/68ab97b3-e90b-4c3a-a83f-6e46232e3844',
 'dg.MD1R/691724b3-a3bc-4645-a26b-8838798eaa3b',
 'dg.MD1R/f038c3c4-383d-4a12-98bc-cb745b6d174f',
 'dg.MD1R/043a32ac-bc71-4b48-8e8b-72c76c4dd187',
 'dg.MD1R/979f0dbb-97d8-4eb1-9b60-9742080a686c',
 'dg.MD1R/ac8891a3-cf5b-45eb-a23d-d5f1fff409a6',
 'dg.MD1R/1100ed2e-f455-4872-94d1-0bdb3b62bd7d',
 'dg.MD1R/83899422-26c5-4793-9267-d6f58e6c7480',
 'dg.MD1R/054093fd-90ef-47d9-8823-bd12a6daa322',
 'dg.MD1R/d7d0257f-bd43-4b46-932e-b221321934c1',
 ...]
In [199]:
## Elastic search is handy for accessing files for a cohort since object_ids associated with each study or case are joined to the table 
case_object_ids = []
for case in case_data:
    if 'object_id' in case:
        object_id_list = case['object_id']
        for object_id in object_id_list:
            case_object_ids.append(object_id)
display(case_object_ids)
['dg.MD1R/b5cebe0d-907a-4ab2-aef7-b96cebd7d448',
 'dg.MD1R/1355977e-81ed-4ac5-934d-b20407eab280',
 'dg.MD1R/ed0954b6-f76d-4e74-bb7e-3818d69d8001',
 'dg.MD1R/109daa01-5969-4c2c-bb44-25efe88856e6',
 'dg.MD1R/6f330232-cee6-4fef-8d11-e5d04a5951f0',
 'dg.MD1R/c32a89c9-eca8-426a-a6b3-a22b7700ecbc',
 'dg.MD1R/9e1a772a-f5b8-485d-9f30-600e1c052313',
 'dg.MD1R/8ca85564-7d98-4312-8507-64f173333da9']

Sending aggregations with guppy¶


  • Guppy has the ability to return some useful statistics (e.g., histograms) using aggregations.
  • The Gen3Query.graphql_query() function can be used to send aggregations and other more complex queries that the basic Gen3Query.query() function can't support: https://github.com/uc-cdis/gen3sdk-python/blob/8196cf4b76a65d0b9b31c8637a18dfac2a911b56/gen3/query.py#L112
In [210]:
## A more complex example using Python requests
query_txt = """{
    _aggregation {
        case {
          sex {
            histogram {
              key
              count
            }
          }
          race {
            histogram {
              key
              count
            }
          }
          ethnicity {
            histogram {
              key
              count
            }
          }
        }
    }
}"""
response = query.graphql_query(query_string=query_txt)
display(response)
{'data': {'_aggregation': {'case': {'sex': {'histogram': [{'key': 'Female',
       'count': 5931},
      {'key': 'Male', 'count': 5109},
      {'key': 'Not Reported', 'count': 2},
      {'key': 'no data', 'count': 857}]},
    'race': {'histogram': [{'key': 'White', 'count': 5112},
      {'key': 'Black or African American', 'count': 3082},
      {'key': 'Not Reported', 'count': 1814},
      {'key': 'Asian', 'count': 640},
      {'key': 'Other', 'count': 117},
      {'key': 'American Indian or Alaska Native', 'count': 41},
      {'key': 'Native Hawaiian or other Pacific Islander', 'count': 29},
      {'key': 'no data', 'count': 1064}]},
    'ethnicity': {'histogram': [{'key': 'Not Hispanic or Latino',
       'count': 10061},
      {'key': 'Hispanic or Latino', 'count': 923},
      {'key': 'Not Reported', 'count': 58},
      {'key': 'no data', 'count': 857}]}}}}}
In [212]:
## Here is an example simple script for sending a basic aggregation request that will return the data as a DataFrame ("TSV")
## https://github.com/cgmeyer/gen3sdk-python/blob/5fd6b868374f622221c0c0173a0d9489b190facd/expansion/expansion.py#L3511

data = exp.guppy_aggregation(node='case', prop='race', format='TSV')
display(data)
Requesting 'https://data.midrc.org/guppy/graphql': {'query': '{_aggregation {case {race {histogram {key count} }}}}', 'variables': None}
key count
0 White 5112
1 Black or African American 3082
2 Not Reported 1814
3 Asian 640
4 Other 117
5 American Indian or Alaska Native 41
6 Native Hawaiian or other Pacific Islander 29
7 no data 1064
In [190]:
## A more complex example using Python requests
query_txt = """{
    _aggregation {
        case {
          sex {
            histogram {
              key
              count
            }
          }
          race {
            histogram {
              key
              count
            }
          }
          ethnicity {
            histogram {
              key
              count
            }
          }
        }
    }
}"""
query_json = {"query": query_txt}
guppy_url = "{}/guppy/graphql".format(api)
response = requests.post(guppy_url, json=query_json, auth=auth)
display(json.loads(response.text)['data']['_aggregation']['case'])
{'sex': {'histogram': [{'key': 'Female', 'count': 5931},
   {'key': 'Male', 'count': 5109},
   {'key': 'Not Reported', 'count': 2},
   {'key': 'no data', 'count': 857}]},
 'race': {'histogram': [{'key': 'White', 'count': 5112},
   {'key': 'Black or African American', 'count': 3082},
   {'key': 'Not Reported', 'count': 1814},
   {'key': 'Asian', 'count': 640},
   {'key': 'Other', 'count': 117},
   {'key': 'American Indian or Alaska Native', 'count': 41},
   {'key': 'Native Hawaiian or other Pacific Islander', 'count': 29},
   {'key': 'no data', 'count': 1064}]},
 'ethnicity': {'histogram': [{'key': 'Not Hispanic or Latino', 'count': 10061},
   {'key': 'Hispanic or Latino', 'count': 923},
   {'key': 'Not Reported', 'count': 58},
   {'key': 'no data', 'count': 857}]}}
In [214]:
## Count the number of files in each project
files_by_project = """
{
  _aggregation {
    imaging_data_file {
      project_id {
        histogram {
          key
          count
        }
      }
    }
  }
}"""
response = query.graphql_query(files_by_project)
display(response)
{'data': {'_aggregation': {'imaging_data_file': {'project_id': {'histogram': [{'key': 'Open-A1',
       'count': 59611},
      {'key': 'Open-R1', 'count': 25003},
      {'key': 'Open-A1_PETAL_REDCORAL', 'count': 6037}]}}}}}

Use the guppy download endpoint to access ElasticSearch tables.¶


  • Tables of data from ES can be exported from the data exploration app (https://data.midrc.org/explore) by using the "Download Table" button.
  • To get these sorts of tables using the API, you can use the guppy download function: https://github.com/uc-cdis/gen3sdk-python/blob/8196cf4b76a65d0b9b31c8637a18dfac2a911b56/gen3/query.py#L146
In [232]:
## This query gets all the imaging studies of modality "CT"

query.raw_data_download(
                    data_type="imaging_study",
                    fields=[
                        "study_uid",
                        "project_id",
                        "study_description",
                        "body_part_examined",
                        "case_ids",
                        "object_id"
                    ],
                    filter_object={"=": {"study_modality": "CT"}}
                )
Out[232]:
[{'case_ids': ['10003752-DDqm9M6Q0OeSyzuCJPMw'],
  'study_uid': '2.16.840.1.114274.1818.51777461259609084757651911267630769342',
  'project_id': 'Open-A1',
  'study_description': '"CT ANGIO CHEST W"',
  'object_id': ['dg.MD1R/556fd113-1d96-489f-a717-6cead2b584bb',
   'dg.MD1R/3bb14f1d-aa48-4e4b-a05a-010393240399',
   'dg.MD1R/3a80ee80-e9be-452d-88d6-7bec53d01d0b',
   'dg.MD1R/91bc7178-4575-4339-af46-82e13baf2949',
   'dg.MD1R/1863fdd5-fdcd-4a48-af18-0770a7773164',
   'dg.MD1R/e1634f84-efd3-4bab-9f16-094ff491d32f',
   'dg.MD1R/bc8cbf19-cbb3-43bb-8594-ff184c0a65fb',
   'dg.MD1R/3a8a86b6-b228-432a-8739-a450f70c942d',
   'dg.MD1R/1ee74d88-b0e2-4906-ac53-f86a63baea44',
   'dg.MD1R/f6ceb0c3-e415-495c-a086-42d418005441']},
 {'case_ids': ['419639-010098'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.804433314179349312091576769322',
  'project_id': 'Open-R1',
  'study_description': 'CT CHEST PULMONARY EMBOLISM (CTPE)',
  'object_id': ['dg.MD1R/b3e7f7f9-7823-4a06-9963-a74345e00081',
   'dg.MD1R/f98d4f54-0955-4946-a728-4391692af9f1',
   'dg.MD1R/55564841-236b-4e9b-a63e-cc94e82f3b0c',
   'dg.MD1R/5f894e32-be9e-40b8-a584-975ddaa42318',
   'dg.MD1R/c447e336-696a-488a-862b-1d79c36348e5',
   'dg.MD1R/4a76c36c-32a8-403a-9106-f1bf393758bd',
   'dg.MD1R/ab8ca25b-93fa-4a40-9fa7-3a862fb93d29',
   'dg.MD1R/ee283eb0-1ac1-4cf2-92e3-d3a61dad23d1',
   'dg.MD1R/8246e663-dcfa-4732-b80c-b3920ba42b55']},
 {'case_ids': ['10000364-206421'],
  'study_uid': '1.3.6.1.4.1.5962.99.1.3492129840.282517718.1618866294295.15942.0',
  'project_id': 'Open-A1',
  'study_description': 'Head^HEAD (Adult)',
  'object_id': ['dg.MD1R/07d10b9e-5bcd-4b03-86da-75cef411a230',
   'dg.MD1R/8522ea81-9b14-4877-b455-66a63c9f43b3',
   'dg.MD1R/72725cc9-4d7d-41c9-a052-c00385cdcc22',
   'dg.MD1R/15a058a6-8331-4493-9fdb-76517392f1e9',
   'dg.MD1R/35d73f75-c80e-4407-9adc-903c7ade327d']},
 {'case_ids': ['10003752-7EoVbIAt0W1sx0wNkjKsQ'],
  'study_uid': '2.16.840.1.114274.1818.554273312396014442418013776475710842764',
  'project_id': 'Open-A1',
  'study_description': 'CT CHEST WO',
  'object_id': ['dg.MD1R/854d603f-44e2-4b88-899d-955300a30b76',
   'dg.MD1R/4b0749c5-4641-428a-a0d8-b40618e8ac09',
   'dg.MD1R/4474fcff-9b52-408e-86da-dcf369b437b4',
   'dg.MD1R/890cfefc-af49-4998-a02b-ea0e4b09868b',
   'dg.MD1R/2be6c716-f94a-4ae5-8346-f719fdd622b1',
   'dg.MD1R/4d69da2d-f162-442d-b622-6f32fc5f5329',
   'dg.MD1R/f866e1c8-7397-43c7-9cba-fc202ae7e1cb',
   'dg.MD1R/eeecd143-4c65-4e89-b876-5bbe67329005',
   'dg.MD1R/890e2dd2-fa14-4f20-a95a-963e876efa9d']},
 {'case_ids': ['419639-010771'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.220366798732491376023882886112',
  'project_id': 'Open-R1',
  'study_description': 'CT CHEST PULMONARY EMBOLISM (CTPE)',
  'object_id': ['dg.MD1R/ae712351-64c9-46ff-9fe7-5bf3d8e5182d',
   'dg.MD1R/6819fa76-0eb7-461c-a533-e92c0de1e648',
   'dg.MD1R/0ab0b67c-5d8a-40d1-a313-c4e052a890b7',
   'dg.MD1R/42a441da-7a09-4cfd-bf61-4c5d1fa7971f',
   'dg.MD1R/217b9a20-2b97-489f-9a49-fe1a7c7ff06e',
   'dg.MD1R/cd78751e-7767-481c-84e9-b49c162dba1a',
   'dg.MD1R/0b81e725-f94e-42e8-a5c8-a185d7a6d7b5',
   'dg.MD1R/b5f52f26-1f71-455e-a63e-e8e06c91e4e5']},
 {'case_ids': ['419639-000782'],
  'study_uid': '1.2.826.0.1.3680043.10.474.419639.113732345190386825100952792264',
  'project_id': 'Open-R1',
  'study_description': 'CT CHEST PULMONARY EMBOLISM (CTPE)',
  'body_part_examined': ['CHEST'],
  'object_id': ['dg.MD1R/2edb6b50-96af-4f86-ac61-e776b8280135',
   'dg.MD1R/5a9cc20e-d242-4675-ac24-13b0ea493533',
   'dg.MD1R/0613fad8-c827-4b86-9ee7-7ecfab263af1',
   'dg.MD1R/1e68b4f8-9edb-4357-bd56-d066b7475358',
   'dg.MD1R/54d93dd4-eb41-4b0a-9445-1676483e7b5a',
   'dg.MD1R/d664c109-5373-491c-8042-3040b2d6d54a',
   'dg.MD1R/e84df3ae-598a-4f47-ba6d-7b9f3dfc5922']},
 {'case_ids': ['10000364-1323552'],
  'study_uid': '2.16.840.1.114274.1818.50249379688538416274999730733663772051',
  'project_id': 'Open-A1',
  'study_description': 'CT CHEST ABDOMEN PELVIS W CONTRAST (TRAUMA)',
  'object_id': ['dg.MD1R/e0a2c2fc-964f-4a0c-b844-fb01251b7720',
   'dg.MD1R/053edf22-856b-41c6-a4f8-f7355c50ea86',
   'dg.MD1R/4aa0e548-6f84-424d-80be-bbc3199c743d',
   'dg.MD1R/c564319d-86b6-4c1b-9317-682b72702848',
   'dg.MD1R/346fe021-1518-4277-ae97-012c21ca2af0',
   'dg.MD1R/adc09b6d-f3e2-495d-98b3-dcececbd30ec',
   'dg.MD1R/e717c25f-036b-49f0-b158-6d356d3b9cd5',
   'dg.MD1R/8e132f94-94b3-4226-9125-878524513f3e',
   'dg.MD1R/1536a620-62dc-4cd2-9eec-583ef4b27556',
   'dg.MD1R/5e026d42-5347-43dd-b82a-43e5cefbd2f7',
   'dg.MD1R/3a1435ed-f9a8-49dd-817e-e16298d5cf64',
   'dg.MD1R/6255ad1a-594b-40c9-a8af-622a0f7170bc',
   'dg.MD1R/cd91f3cf-b252-4dbb-9481-ebd01c117273',
   'dg.MD1R/7cd884a0-334a-402a-93e5-2e71f6b72b65',
   'dg.MD1R/132cff67-91b0-455a-bcf0-62cad9b0aa70',
   'dg.MD1R/0c874d42-fcfb-4389-9896-42e28d1e14fa',
   'dg.MD1R/625cd811-c496-4ec6-800b-3c3e697c715a']},
 {'case_ids': ['10000364-11386'],
  'study_uid': '2.16.840.1.114274.1818.512088006644096519010865547107149277361',
  'project_id': 'Open-A1',
  'study_description': 'CT LEVEL CHEST ABDOMEN PELVIS W ED',
  'object_id': ['dg.MD1R/dac5f633-8185-45cc-b58e-50dad8889e05',
   'dg.MD1R/709235e5-dcda-49b7-b661-953cbd27a26e',
   'dg.MD1R/b67b1f13-a953-4902-9b3d-8d3e5a8bef62',
   'dg.MD1R/fec95d1a-a34c-49ba-9fe3-173c4f8fd45f',
   'dg.MD1R/05c1cb14-b556-47dd-984c-d701b19bee24',
   'dg.MD1R/3be8db42-31ec-40a4-a1dd-49852a3dfb2e',
   'dg.MD1R/8a43afb4-474f-4c22-a6aa-c9e7df5e5450',
   'dg.MD1R/bc12bef6-f1d4-4165-9877-196f4cf83540',
   'dg.MD1R/4c5dfe97-b5e2-4b1a-b839-d0f4379b9c4f',
   'dg.MD1R/ab8fdc25-3461-4441-b2c3-c71ec6cc703c',
   'dg.MD1R/2d438cfc-fe2a-45c3-8ef6-84da69df7ddf',
   'dg.MD1R/e4cf357d-4414-4ee9-b621-779795820454',
   'dg.MD1R/d49f08c1-0e54-4e38-87af-a7680e8c2424',
   'dg.MD1R/7a92de1a-d8d5-4578-a3bf-6c0a54c96c0e',
   'dg.MD1R/f79e8c37-3601-45f1-bdb4-4786d91efefe',
   'dg.MD1R/12ece514-0b27-4a0d-9d6d-3030871f1d97',
   'dg.MD1R/41e7c04f-f632-4afb-9eca-fc3d791d0012',
   'dg.MD1R/6e4651d9-a66d-4325-8856-d7f9a2a38a30']},
 {'case_ids': ['10008204-hozU7Z5LwU2rA2WGk3fuNg'],
  'study_uid': '2.16.840.1.114274.1818.56239264145578181001864632229166092423',
  'project_id': 'Open-A1',
  'study_description': '"CHEST W/O CONTRAST (CT)-CS"',
  'object_id': ['dg.MD1R/79de7110-63b3-4cca-9c25-a9a4bd533e25',
   'dg.MD1R/8858e790-8648-451d-b1e0-b165db7752b5',
   'dg.MD1R/db40ea60-2955-4cb7-bcdb-3167b6dcf223',
   'dg.MD1R/6a2b4220-e601-4acf-a249-75308e280e61',
   'dg.MD1R/42961e97-c651-4257-8300-c88066bffc13',
   'dg.MD1R/411b20c3-7187-4f48-81b0-1ba1d5834d6b',
   'dg.MD1R/a0b139d3-74f5-4670-9a70-1a52a359a8e3']},
 {'case_ids': ['10000364-5953493'],
  'study_uid': '2.16.840.1.114274.1818.510025634945909270015180527370270184880',
  'project_id': 'Open-A1',
  'study_description': 'Thorax^CT_CA_WO (Adult)',
  'object_id': ['dg.MD1R/1c952c18-47fc-4c83-80cd-4ac3b6f5c3b3',
   'dg.MD1R/72e64d97-b35c-4ac3-abdf-ebaa6591c9bd',
   'dg.MD1R/dcc9d960-349e-43f9-87dc-38cfb415f756',
   'dg.MD1R/3c670dcc-f08c-45ea-9abe-e75cd78c3b55',
   'dg.MD1R/880b6b99-647b-4dbc-8335-91dd19614500']},
 {'case_ids': ['10000364-5823840'],
  'study_uid': '2.16.840.1.114274.1818.54890006036162651374264989034271960451',
  'project_id': 'Open-A1',
  'study_description': 'Vascular^DISSECTION_CTA_CAP (Adult)',
  'object_id': ['dg.MD1R/427a78df-a404-4c1d-9be0-a3bab382095c',
   'dg.MD1R/97291a1c-e7c5-409b-b743-30dcbe1d85c7',
   'dg.MD1R/f5fd7a43-8a57-43a7-9de8-aea6e6846a30',
   'dg.MD1R/20cc5c41-0737-4a23-bdd3-77e286d7dac6',
   'dg.MD1R/76d67794-89a3-41c3-bb7a-c518eac53689',
   'dg.MD1R/e971145c-9a61-4534-b2ac-da43cdd9016c',
   'dg.MD1R/facb4e0a-e27e-46b6-90c4-8104915b6053',
   'dg.MD1R/c55b1394-6d65-47e5-b5c2-b24640bd6df2']},
...]
In [227]:
## Here is an example getting all the cases in a particular project between ages of 45 and 47

query.raw_data_download(
                    data_type="case",
                    fields=[
                        "submitter_id",
                        "project_id",
                        "race",
                        "sex",
                        "ethnicity",
                        "age_at_index",
                        "object_id"
                    ],
                    filter_object={"AND": [{">=": {"age_at_index": 45}},
                                           {"<=": {"age_at_index": 47}},
                                           {"=": {"project_id": "Open-A1"}}]},
                    sort_fields=[{"submitter_id": "asc"}],
                    accessibility="accessible"
                )
Out[227]:
[{'ethnicity': 'Not Hispanic or Latino',
  'race': 'White',
  'project_id': 'Open-A1',
  'sex': 'Male',
  'age_at_index': 47,
  'submitter_id': '10000364-206421',
  'object_id': ['dg.MD1R/f99825c3-68ea-4a2a-a466-9052fbe6d80a',
   'dg.MD1R/44348bdb-a831-44ec-907c-1f7002bbeff9',
   'dg.MD1R/cd02ad3c-208e-47d3-8c49-132e64939458',
   'dg.MD1R/ad7ca02e-44fc-48ba-92da-e9d2ba12941a',
   'dg.MD1R/6de50ee5-f71f-4400-98ea-416fb8e909b8',
   'dg.MD1R/72725cc9-4d7d-41c9-a052-c00385cdcc22',
   'dg.MD1R/a2c28f50-a4e1-4d19-9fe1-915060baacd0',
   'dg.MD1R/2a36fa46-c03f-4580-84af-c43d2a5ae9a6',
   'dg.MD1R/98b37604-d2b7-486b-a49b-4e3864f199b6',
   'dg.MD1R/8522ea81-9b14-4877-b455-66a63c9f43b3',
   'dg.MD1R/fd6dd641-f65c-473c-bdd7-518afa0dae11',
   'dg.MD1R/402505b9-dd29-430f-aa3e-afe1d3115d40',
   'dg.MD1R/15a058a6-8331-4493-9fdb-76517392f1e9',
   'dg.MD1R/d4d46fcd-0d3c-4e2c-b35c-13627c99b4f9',
   'dg.MD1R/07d10b9e-5bcd-4b03-86da-75cef411a230',
   'dg.MD1R/6139aa55-923b-4318-9d4d-a1e2ce0b7a7c',
   'dg.MD1R/1d024937-2866-417e-b176-7f12ac7dfbc5',
   'dg.MD1R/35d73f75-c80e-4407-9adc-903c7ade327d',
   'dg.MD1R/c6ce4e7d-a73d-4d88-9687-37f1e15b0d40']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'Black or African American',
  'project_id': 'Open-A1',
  'sex': 'Female',
  'age_at_index': 46,
  'submitter_id': '10000364-1288962',
  'object_id': ['dg.MD1R/1b7a4aca-a724-45fa-bdd9-7894d1de6259',
   'dg.MD1R/5f362efd-418b-47e9-bdd8-b88ec801b0ca']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'White',
  'project_id': 'Open-A1',
  'sex': 'Male',
  'age_at_index': 46,
  'submitter_id': '10008204-oLAFzo7d0MacboPylxhg',
  'object_id': ['dg.MD1R/d3ab6406-1e72-4c4e-a65a-a5ca3e3ee63c',
   'dg.MD1R/df6474ba-8a2a-4a5f-a0af-45df2b0f509f']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'Black or African American',
  'project_id': 'Open-A1',
  'sex': 'Female',
  'age_at_index': 45,
  'submitter_id': '10003752-7abGbWvV6UaEc0nAAWvfYQ',
  'object_id': ['dg.MD1R/8d741781-36f6-4a2b-9b63-ba6d15976707',
   'dg.MD1R/4571c711-58e9-4079-bbb0-4b18ed698b29']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'White',
  'project_id': 'Open-A1',
  'sex': 'Male',
  'age_at_index': 45,
  'submitter_id': '10000364-2732012',
  'object_id': ['dg.MD1R/976498e7-cda1-4e2e-bf3f-9102cba06b97',
   'dg.MD1R/b3aed8e7-b25b-4e19-ac03-eb5d411b0d6b',
   'dg.MD1R/c2a48bcf-4ae6-4300-bec3-89fdb0455405',
   'dg.MD1R/27f213b7-b40e-4a08-8b3a-b1d116635769',
   'dg.MD1R/a41440f7-1036-4204-933a-6bd65c83b791',
   'dg.MD1R/84ef98c9-0d38-4184-ae2d-76199521d1c2',
   'dg.MD1R/bf8f4f7e-5d4f-4991-8bb1-5f01522925ce',
   'dg.MD1R/3396bc04-a787-40d9-8dca-cd3694d1cc16',
   'dg.MD1R/e67c88c8-e9d0-40de-9bd3-c1cc2fd08434',
   'dg.MD1R/522d277b-3cb2-4c43-9072-c08426ac0404',
   'dg.MD1R/884e5e86-1a1c-4636-a8b4-2b3def5dda3b',
   'dg.MD1R/830f5d85-331b-4ba8-9133-8af912fac3d0',
   'dg.MD1R/b6999741-0c06-4685-ba41-fc691bf8a3cb',
   'dg.MD1R/ac4bcdd3-2d24-4994-92b3-6b702cb47a40',
   'dg.MD1R/e6a83505-a2a2-44e3-b729-dac2f535b4b8',
   'dg.MD1R/f92fcf04-564e-4123-9ca3-725eab953e10']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'Black or African American',
  'project_id': 'Open-A1',
  'sex': 'Female',
  'age_at_index': 47,
  'submitter_id': '10000364-1510060',
  'object_id': ['dg.MD1R/fc4f55b9-a2e8-4be5-be02-efb27d259885',
   'dg.MD1R/35174e99-e613-4212-9e7d-c7c561741ec5',
   'dg.MD1R/9f5e3657-538e-455f-94b9-d1a86591069f',
   'dg.MD1R/182e71a9-dd68-4228-9d3a-c6afe79fd239',
   'dg.MD1R/bf369fb2-a3a7-42d3-a378-020c360203f3',
   'dg.MD1R/2ecf7cbe-350f-4cdc-bf5d-ef4bf82ebd1b',
   'dg.MD1R/b278bcf2-18d4-45c4-b7fd-87cd1df87155',
   'dg.MD1R/12acee8c-1276-452f-83f9-88a59cfed5c9',
   'dg.MD1R/2a91db36-38bb-46de-b1fa-4fb347fd890c',
   'dg.MD1R/96ceee5b-f4d1-4f80-9005-67b7151ccf1b',
   'dg.MD1R/2301720b-fafd-48b4-8e7e-52563071463b',
   'dg.MD1R/0ab2af7d-777a-4e96-ba0a-70ad379b5599',
   'dg.MD1R/b2f96e1f-3ed9-4f1f-b67d-7943e195fe9f',
   'dg.MD1R/8cc12391-e11b-47ac-8830-ca7c824f3479',
   'dg.MD1R/f362c2e4-914c-44ee-8b62-f16b2eb4fa20',
   'dg.MD1R/8327e539-564a-49c8-a0c6-4b9d2fba5f71',
   'dg.MD1R/d9e4eeca-233a-43fb-b63f-405b60418bd6',
   'dg.MD1R/3b94939c-2504-4aee-93ca-2a3c04c8a9b0']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'White',
  'project_id': 'Open-A1',
  'sex': 'Male',
  'age_at_index': 45,
  'submitter_id': '10000364-1125559',
  'object_id': ['dg.MD1R/e975c2ea-a82d-4328-88cf-ded5cc5a16fc']},
 {'ethnicity': 'Hispanic or Latino',
  'race': 'Not Reported',
  'project_id': 'Open-A1',
  'sex': 'Male',
  'age_at_index': 46,
  'submitter_id': '10000364-2055101',
  'object_id': ['dg.MD1R/6d7367cc-af4d-471d-a7ef-0e84b5be39d3',
   'dg.MD1R/7d420cf8-fb36-41f9-8c17-a8d3a949abfc',
   'dg.MD1R/a7959efd-6119-4c93-89a2-32221222990c',
   'dg.MD1R/05f23578-a3b5-4264-9291-9be1e5f773a9',
   'dg.MD1R/d4f1ea3c-c15a-4f24-b8a8-f1fe9dc126f5',
   'dg.MD1R/c3a04500-36be-41e8-b034-6e02ee94ee19',
   'dg.MD1R/8fb4bcbb-44a7-4e59-900f-84f20838f489']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'Black or African American',
  'project_id': 'Open-A1',
  'sex': 'Female',
  'age_at_index': 46,
  'submitter_id': '10003752-GvRyJHJcWUhA5O3cgWyQ',
  'object_id': ['dg.MD1R/bd662878-1524-4631-8ae7-e2315c40c5a0']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'Black or African American',
  'project_id': 'Open-A1',
  'sex': 'Male',
  'age_at_index': 46,
  'submitter_id': '10003752-kfqUyPohA0OcAh700bIyfQ',
  'object_id': ['dg.MD1R/49dea332-a36b-42aa-ae08-adbe0d5b1e41',
   'dg.MD1R/59bbb918-b207-41a0-8e77-9b1cd42d90ca',
   'dg.MD1R/60ed8ff1-1177-433d-8975-2fce318113fb',
   'dg.MD1R/972441fe-ebe0-45b1-8f24-a3a746976756',
   'dg.MD1R/aa9d8b91-eaa2-4dd1-8f8c-8f9cfc545a2d',
   'dg.MD1R/d01cd6aa-977f-4450-84f5-5c0b7f089700',
   'dg.MD1R/56685e79-52dd-44bc-a3ed-84fc2cb98e79',
   'dg.MD1R/067b5cc2-4a3e-4c36-a707-da24b69a0f5b',
   'dg.MD1R/4c34d30d-7cb3-4651-ba73-f5e9ff14587f',
   'dg.MD1R/cfce0105-32f8-4bea-b134-44a44b2903e6',
   'dg.MD1R/41a06a21-46ab-4b97-a2c7-c9d252c51010',
   'dg.MD1R/1f9c6ec7-def8-49a8-82cb-854bc21032d8',
   'dg.MD1R/ee70e04a-99bc-47b1-89a0-486b56ad21dd',
   'dg.MD1R/1a39576c-416f-42b1-8118-3dcfcbe8bd77',
   'dg.MD1R/9591ff07-df17-4eaf-ba90-94fbb71fe27e',
   'dg.MD1R/24f88bb3-27d0-4847-a168-ee397ebe57a6',
   'dg.MD1R/89288db6-1000-48a4-8f17-07ab40873fe7',
   'dg.MD1R/158b54dd-1007-4e5c-92be-92d102426bf2',
   'dg.MD1R/8430f9c3-0c54-4535-a1c3-b56eb32b6c23',
   'dg.MD1R/a3a03c06-1735-49db-8c47-295636244989',
   'dg.MD1R/637f56c9-b7f3-42b7-868c-6bfe316143cf',
   'dg.MD1R/d11c4da0-d2f9-44b8-93ce-c7b4f908e019',
   'dg.MD1R/83570ed3-3ecd-4c3d-a939-c036f2fa9ae7',
   'dg.MD1R/70fd72d1-053c-48ff-9890-52c51e4584d3',
   'dg.MD1R/ba1a2229-09dd-471b-8a07-7f074bac872c',
   'dg.MD1R/a591b056-8eb4-46e5-a52d-b525a65bfa87',
   'dg.MD1R/1988219a-f33b-4964-bd29-552f1b2d1d33',
   'dg.MD1R/fa941ff0-8f43-4ecb-887a-6ec0d2b7205c',
   'dg.MD1R/1f050a8e-dc8a-4625-9987-5b6928bd7ecf',
   'dg.MD1R/b2f7354e-287c-4306-8cb2-1988704f06b8',
   'dg.MD1R/9d8edcac-d987-4c1a-a871-542f66eb22b3',
   'dg.MD1R/113c39ff-649c-43b7-884c-81ba911064b9',
   'dg.MD1R/dc07e982-5756-4f8c-8bc6-01b4da804d0d',
   'dg.MD1R/7840ec13-122e-42b1-9fdb-e03a1c223fa8',
   'dg.MD1R/7e63454c-d104-4122-8602-96d32c825eb4',
   'dg.MD1R/3ab77fad-48b7-4827-b0d7-52e9b4a5e49f',
   'dg.MD1R/6d7dac18-c8c8-4203-aefe-9676e11d2bf2',
   'dg.MD1R/80cadfc3-953a-4834-851e-78bf80bea6f3',
   'dg.MD1R/e7cdcfff-02ea-4e16-b876-3cb19dd38c36',
   'dg.MD1R/4a407f5f-d5be-483b-be8c-34d3a7a334e5',
   'dg.MD1R/70abc856-93f6-476e-a942-e138798ccad2',
   'dg.MD1R/b638bb53-d321-4f4f-9cc3-874639a2b780',
   'dg.MD1R/ade055c1-581a-4495-a01f-4be7a317cce1',
   'dg.MD1R/284b3e85-0456-4f86-a5d7-110bf5f698fe',
   'dg.MD1R/c11829f1-b034-4ff5-8bfb-fdd165f1a367',
   'dg.MD1R/7d181850-cfb4-4f02-b8be-f841efb0233d',
   'dg.MD1R/1b1b244f-15aa-4276-8e31-0938349c074c',
   'dg.MD1R/ec84d326-f56a-4a17-87cf-10b46b82b917',
   'dg.MD1R/128b24ab-9dae-4e8e-8090-e8f9febb1269',
   'dg.MD1R/9eea3203-2784-48a4-847c-4fd561cc9271',
   'dg.MD1R/ce13a15c-9a40-4cb5-ad65-b11e7a32d82f',
   'dg.MD1R/bf8cbdad-fb49-402d-b084-1d837cbb30e3',
   'dg.MD1R/92ab7644-a933-4136-9803-aa3336f00675',
   'dg.MD1R/9ea06b04-583b-4aef-a793-6a9c17cdc420',
   'dg.MD1R/a2b04e43-ebf9-4d6a-a659-9cafb56d4de8',
   'dg.MD1R/3abb682e-1ac5-4f01-aed2-9838ddbc25d0',
   'dg.MD1R/1f372ca5-6a41-4880-b427-b5706c4209cb',
   'dg.MD1R/4ab197a7-859f-4965-8ee2-16b016e72712',
   'dg.MD1R/e2574eff-c993-4125-95c1-def247c94386',
   'dg.MD1R/4e371fd8-9f77-492c-8725-7ca9cb1d4276',
   'dg.MD1R/365fb30c-04c5-48b2-8f66-c95fa9d11af9',
   'dg.MD1R/ff17a9ac-a959-4613-8fec-3e0d1bbee885',
   'dg.MD1R/a4175aa0-318d-4025-89dd-b0ba48d6830e',
   'dg.MD1R/a287101d-2bc8-4f5f-bd07-e949af4731ba',
   'dg.MD1R/bb288535-81f4-4a16-84fe-c7bb61a3ddc3',
   'dg.MD1R/c6b5010c-47f2-4a50-a4d3-77425ef4b55c',
   'dg.MD1R/74a571ed-c9d0-45ee-9eee-4f11f1a0b81e',
   'dg.MD1R/f39d679a-a9ac-4572-9c81-618155b710a1',
   'dg.MD1R/bccb20c7-2479-432e-8969-c3a7070e5e78',
   'dg.MD1R/17600984-a72a-4464-bf26-045ce3c15724',
   'dg.MD1R/e4644d7a-b95e-4733-88f9-330c49d3237f',
   'dg.MD1R/215222d2-6633-4f82-95b6-492af82aeb8b',
   'dg.MD1R/0074f951-fbc7-4014-8887-321b1ee10617',
   'dg.MD1R/5523d1ef-b729-419b-9669-6700b54386b9',
   'dg.MD1R/3bc11ee3-3176-4c63-8ebe-583734732406',
   'dg.MD1R/7d2da3e1-fd1e-4b1b-a763-77dcd8fb8b64',
   'dg.MD1R/710e33ce-5e96-48d2-9732-b0111591a364',
   'dg.MD1R/e86b5ce5-2a0f-49d3-8d22-e6b9714b8dcb',
   'dg.MD1R/d13d9288-9863-498a-9ee8-7952702b4547',
   'dg.MD1R/304b84f2-d294-4466-a05a-cad5bc7c901f']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'White',
  'project_id': 'Open-A1',
  'sex': 'Male',
  'age_at_index': 46,
  'submitter_id': '10000364-1754485',
  'object_id': ['dg.MD1R/57f44b39-3d21-4dec-baf4-94cf5671a607',
   'dg.MD1R/82da8c44-ee2f-4070-9181-947635d5d61e']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'White',
  'project_id': 'Open-A1',
  'sex': 'Male',
  'age_at_index': 47,
  'submitter_id': '10000364-1142358',
  'object_id': ['dg.MD1R/cd5f0cf6-283d-4562-aeed-c2c4252878c1',
   'dg.MD1R/bc372d0e-4a62-4865-acda-09bf1cacb864',
   'dg.MD1R/4de635ff-f97c-4801-800b-f1ac765c46e9',
   'dg.MD1R/496633bb-6a73-4d8b-a447-5704d6c6a2f7',
   'dg.MD1R/83466397-53b9-42bc-b0b5-890ed284429a',
   'dg.MD1R/b0887624-8ba0-45da-85bc-d8a2aa66ea22',
   'dg.MD1R/ba3b958e-8319-4e49-8c15-a7af828db2a7',
   'dg.MD1R/32bd26d0-a665-42b8-85d1-92c21418755b',
   'dg.MD1R/497c9374-6dd8-4f7c-ae1f-43d3f9fd2c38',
   'dg.MD1R/36c2aec4-e37f-4a6e-a5f9-50b5ca1aa706',
   'dg.MD1R/d2f604d5-2824-434f-890d-61ba7b9b8c90',
   'dg.MD1R/d6e56bb7-b7f3-4b4a-ae7d-e7a31fe966be',
   'dg.MD1R/da0d1498-8208-481d-9e1e-701f9a2b5c01',
   'dg.MD1R/1515fc76-814c-4128-87a4-25491f6a0591',
   'dg.MD1R/98843c0f-341c-42cb-9345-ac73c8254827',
   'dg.MD1R/4645e105-0163-4323-9bc5-a22d2e357a26',
   'dg.MD1R/93ad657d-86a8-49e9-a4fa-c9848696e2c7',
   'dg.MD1R/633e66fe-38c3-47d9-9ded-cff3160d2ac9']},
 {'ethnicity': 'Not Hispanic or Latino',
  'race': 'Black or African American',
  'project_id': 'Open-A1',
  'sex': 'Female',
  'age_at_index': 47,
  'submitter_id': '10003752-w3uhjBebLUeBwtJdDSIA',
  'object_id': ['dg.MD1R/73b1cfd2-984a-4177-bfbb-cf0c97c6b546',
   'dg.MD1R/3f44c782-2987-44bf-b949-d180632e7a87',
   'dg.MD1R/b10ac77a-0168-48c0-b57b-3d52b8a89da8',
   'dg.MD1R/97c0c8cf-3d2e-40ad-8968-373fa9b7e30f',
   'dg.MD1R/46660883-212e-48f9-b99b-40f039aed0b8',
   'dg.MD1R/2fcd170f-9a3a-40fd-917c-1bd055fef603',
   'dg.MD1R/3b7e4793-b68c-449c-8aaa-f3bd7195459c',
   'dg.MD1R/da2bf679-3e4b-468c-8c28-8d0b28d708f8',
   'dg.MD1R/836f6561-b4a5-4d53-b58a-9f22b888d2ea',
   'dg.MD1R/cd13b0d5-e532-4d97-b20b-354850656a8f',
   'dg.MD1R/460d77d5-0f29-4154-b504-b54c8311f69e',
   'dg.MD1R/6ee2b8e2-eab7-45e7-9187-3f1c74643761',
   'dg.MD1R/6704e8fb-45b7-4aec-baf6-16169c485913',
   'dg.MD1R/27904845-cf3a-4673-ac8e-21c2c7106da0',
   'dg.MD1R/dbfefb1c-c2ca-4212-9a5c-d6c5f442e6f5',
   'dg.MD1R/0d386044-a317-4567-b60b-a03adc051704',
   'dg.MD1R/b7ee26a7-95e0-4098-97d5-e1b30e6a66c4']},
 {'ethnicity': 'Hispanic or Latino',
  'race': 'White',
  'project_id': 'Open-A1',
  'sex': 'Female',
  'age_at_index': 47,
  'submitter_id': '10000364-601680',
  'object_id': ['dg.MD1R/df082157-33cb-464d-a899-4f0c2d73d754',
   'dg.MD1R/98ca782c-b0e9-413d-9a16-f4b06a28ef15']},
   ...]