Data is a critical component of decision making, helping businesses and organizations gain key insights and understand the implications of their decisions at a granular level. And visual analytics, in the form of
interactive dashboards and visualizations, are essential tools for anyone—from students to CEOs—who needs to analyze data and tell stories with data. Public data sets are ideal resources to tap into to create data visualizations. With the information provided below, you can explore a number of free, accessible data sets and begin to create your own analyses. The following COVID-19 data visualization is representative of the the types of visualizations that can be created using free public
data sets. Explore it and a catalogue of free data sets across numerous topics below. Health dashboards can be used to highlight key metrics including: changes in a population’s health over time, how people choose to receive healthcare, or urgent public health information, such as
vaccination rates during a global pandemic. Social Impact dashboards highlight topics related to society as whole - from local to global public policy issues and concerns. Dashboards can be used to visualize the number of police shootings in the United States or analyze anti-refugee sentiment. Social Impact dashboards can
help decision makers understand policy gaps and create solutions to address specific needs. Climate change is one of the most urgent issues of our time. With relevant data, scientists, leaders, and policymakers are able to see trends, make policy recommendations, and share critical findings. Browse the vast
quantity of climate- and environment-related data dashboards through the links below. Try Tableau today for beautiful data visualizations. Try Tableau Today State, local, and federal governments rely on data to guide key decisions and formulate effective policy for their constituents. The data they generate is often in the form of open data sets that are accessible for citizens and groups to download for their own analyses. Browse the list below for a variety of examples. Education dashboards provide educators and others a way to visualize critical metrics that affect student success and the fundamentals of education itself. These dashboards can help inform decision-making at a local, state, and national level. Browse through more education public data sets below. The variety
of data sets outlined below are great resources that showcase that with the right data you can create just about any sort of visualization to tell your own unique story. The variety of data sets outlined below are great resources that showcase that with the right data you can create just about any sort of
visualization to tell your own unique story. The Cancer Genome Atlascancergenomiclife sciencesSTRIDESwhole genome sequencing The Cancer Genome Atlas (TCGA), a collaboration between the National Cancer Institute (NCI) and National Human Genome Research Institute (NHGRI), aims to generate comprehensive, multi-dimensional maps of the key genomic changes in major types and subtypes of cancer. TCGA has analyzed matched tumor and normal tissues from 11,000 patients, allowing for the comprehensive characterization of 33 cancer types and subtypes, including 10 rare cancers. The dataset contains open Clinical Supplement, Biospecimen Supplement, RNA-Seq Gene Expression Quantification, miRNA-Seq Isoform Expression Quantificati... Details → Usage examples
See 29 usage examples → Foldingathome COVID-19 Datasetsalchemical free energy calculationsbiomolecular modelingcoronavirusCOVID-19foldingathomehealthlife sciencesmolecular dynamicsproteinSARS-CoV-2simulationsstructural biology Folding@home is a massively distributed computing project that uses biomolecular simulations to investigate the molecular origins of disease and accelerate the discovery of new therapies. Run by the Folding@home Consortium, a worldwide network of research laboratories focusing on a variety of different diseases, Folding@home seeks to address problems in human health on a scale that is infeasible by another other means, sharing the results of these large-scale studies with the research community through peer-reviewed publications and publicly shared datasets. During the COVID-19 epidemic, Folding@home focused its resources on understanding the vulernabilities in SARS-CoV-2, the virus that causes COVID-19 disease, and working closely with a number of experimental collaborators to accelerate progress toward effective therapies for treating COVID-19 and ending the pandemic. In the process, it created the world's first exascale distributed computing resource, enabling it to generate valuable scientific datasets of unprecedented size. More information about Folding@home's COVID-19 research activities at the Folding@home COVID-19 page. In addition to working directly with experimental collaborators and rapidly sharing new research findings through preprint servers, Folding@home has joined other researchers in committing to rapidly share all COVID-19 research data, and has joined forces with AWS and the Molecular Sciences Software Institute (MolSSI) to share datasets of unprecented side through the AWS Open Data Registry, indexing these massive datsets via the MolSSI COVID-19 Molecular Structure and Therapeutics Hub. The complete index of all Folding@home datasets can be found here. Th... Details → Usage examples
See 24 usage examples → Therapeutically Applicable Research to Generate Effective Treatments (TARGET)cancergenomiclife sciencesSTRIDESwhole genome sequencing Therapeutically Applicable Research to Generate Effective Treatments (TARGET) is the collaborative effort of a large, diverse consortium of extramural and NCI investigators. The goal of the effort is to accelerate molecular discoveries that drive the initiation and progression of hard-to-treat childhood cancers and facilitate rapid translation of those findings into the clinic. TARGET projects provide comprehensive molecular characterization to determine the genetic changes that drive the initiation and progression of childhood cancers.The dataset contains open Clinical Supplement, Biospecimen... Details → Usage examples
See 24 usage examples → Common Crawlencyclopedicinternetnatural language processing A corpus of web crawl data composed of over 50 billion web pages. Details → Usage examples
See 23 usage examples → Gabriella Miller Kids First Pediatric Research Program (Kids First)cancergeneticgenomicHomo sapienslife sciencespediatricSTRIDESstructural birth defectwhole genome sequencing The NIH Common Fund's Gabriella Miller Kids First Pediatric Research Program’s (“Kids First”) vision is to “alleviate suffering from childhood cancer and structural birth defects by fostering collaborative research to uncover the etiology of these diseases and by supporting data sharing within the pediatric research community.” The program continues to generate and share whole genome sequence data from thousands of children affected by these conditions, ranging from rare pediatric cancers, such as osteosarcoma, to more prevalent diagnoses, such as congenital heart defects. In 2018, Kids Fi... Details → Usage examples
See 19 usage examples → NASA Prediction of Worldwide Energy Resources (POWER)agricultureair qualityanalyticsarchivesatmosphereclimateclimate modeldata assimilationdeep learningearth observationenergyenvironmentalforecastgeosciencegeospatialglobalhistoryimagingindustrymachine learningmachine translationmetadatameteorologicalmodelnetcdfopendapradiationsatellite imagerysolarstatisticssustainabilitytime series forecastingwaterweatherzarr NASA's goal in Earth science is to observe, understand, and model the Earth system to discover how it is changing, to better predict change, and to understand the consequences for life on Earth. The Applied Sciences Program serves NASA and Society by expanding and accelerating the realization of societal and economic benefits from Earth science, information, and technology research and development. The NASA Prediction Of Worldwide Energy Resources (POWER) Project, a NASA Applied Sciences program, improves the accessibility and usage NASA Earth Observations (EO) supporting community research in three focus areas: 1) renewable energy development, 2) building energy efficiency, and 3) agroclimatology applications. POWER can help communities be resilient amid observed climate variability through the easy access of solar and meteorological data via a verity of access methods. The latest POWER version includes hourly-based source Analysis Ready Data (ARD), in addition to enhanced daily, monthly, annual, and climatology ARD. The daily time-series spans 40 years for meteorology available from 1981 and solar-based parameters start in 1984. The hourly source data are from Clouds and the Earth's Radiant Energy System (CERES) and Global Modeling and Assimilation Office (GMAO), spanning 20 years from 2001. The hourly data will provide users the ARD needed to model the energy performance of building systems, providing information directly amenable to decision support tools introducing the industry standard EPW (EnergyPlus Weather file). POWER also provides parameters at daily, monthly, annual, and user-defined time periods, spanning from 1984 through to within a week of real time. Additionally, POWER provides are user-defined analytic capabilities, including custom climatologies and climatological-based reports for parameter anomalies, ASHRAE® compatible climate design condition statistics, and building climate zones. The ARD and climate analytics will be readily accessible through POWER's integrated services suite, including the Data Access Viewer (DAV). The DAV has recently been improved to incorporate updated parameter groupings, new analytical capabilities, and the new data formats. POWER also provides a complete API (Application Programming Interface) that allows uses... Details → Usage examples
See 18 usage examples → NEXRAD on AWSagricultureearth observationmeteorologicalnatural resourcesustainabilityweather Real-time and archival data from the Next Generation Weather Radar (NEXRAD) network. Details → Usage examples
See 16 usage examples → NOAA Geostationary Operational Environmental Satellites (GOES) 16, 17 & 18agriculturedisaster responseearth observationgeospatialmeteorologicalsatellite imagerysustainabilityweather NEW GOES-18 Data!!! GOES-18 is now provisional and data has began streaming. Data files will be available between Provisional and the Operational Declaration of the satellite, however, data will have the caveat GOES-18 Preliminary, Non-Operational Data. The exception is during the interleave period when ABI Radiances and Cloud and Moisture Imagery data will be shared operationally via the NOAA Open Data Dissemination Program. GOES satellites (GOES-16, GOES-17, & GOES-18) provide continuous weather imagery and monitoring of meteorological and space environment data across North America. ... Details → Usage examples
See 16 usage examples → Genome Aggregation Database (gnomAD)bioinformaticsgeneticgenomiclife sciencespopulationpopulation geneticsshort read sequencingwhole genome sequencing The Genome Aggregation Database (gnomAD) is a resource developed by an international coalition of investigators that aggregates and harmonizes both exome and genome data from a wide range of large-scale human sequencing projects. The summary data provided here are released for the benefit of the wider scientific community without restriction on use. The v2 data set (GRCh37) spans 125,748 exome sequences and 15,708 whole-genome sequences from unrelated individuals. The v3 data set (GRCh38) spans 71,702 genomes, selected as in v2. Sign up for the gnomAD mailing list here. Details → Usage examples
See 15 usage examples → SpaceNetcomputer visiondisaster responseearth observationgeospatialmachine learningsatellite imagery SpaceNet, launched in August 2016 as an open innovation project offering a repository of freely available imagery with co-registered map features. Before SpaceNet, computer vision researchers had minimal options to obtain free, precision-labeled, and high-resolution satellite imagery. Today, SpaceNet hosts datasets developed by its own team, along with data sets from projects like IARPA’s Functional Map of the World (fMoW). Details → Usage examples
See 15 usage examples → Cell Painting Gallerybioinformaticsbiologycancercell biologycell imagingcell paintingchemical biologycomputer visioncsvdeep learningfluorescence imaginggenetichigh-throughput imagingimage processingimagingmachine learningmedicinemicroscopyorganelle The Cell Painting Gallery is a collection of image datasets created using the Cell Painting assay. The images of cells are captured by microscopy imaging, and reveal the response of various labeled cell components to whatever treatments are tested, which can include genetic perturbations, chemicals or drugs, or different cell types. The datasets can be used for diverse applications in basic biology and pharmaceutical research, such as identifying disease-associated phenotypes, understanding disease mechanisms, and predicting a drug’s activity, toxicity, or mechanism of action (Chandrasekaran et al 2020). This collection is maintained by the Carpenter–Singh lab and the Cimini lab at the Broad... Details → Usage examples
See 16 usage examples → Fly Brain Anatomy: FlyLight Gen1 and Split-GAL4 Imagerybiologyfluorescence imagingimage processingimaginglife sciencesmicroscopyneurobiologyneuroimagingneuroscience This data set, made available by Janelia's FlyLight project, consists of fluorescence images of Drosophila melanogaster driver lines, aligned to standard templates, and stored in formats suitable for rapid searching in the cloud. Additional data will be added as it is published. Details → Usage examples
See 13 usage examples → Allen Cell Imaging Collectionsbiologycell biologycell imagingHomo sapiensimage processinglife sciencesmachine learningmicroscopy This bucket contains multiple datasets (as Quilt packages) created by the Allen Institute for Cell Science (AICS). The imaging data in this bucket contains either of the following:
In addition, many of the datasets include CSVs that contain feature sets related to that data. Details → Usage examples
See 11 usage examples → International Neuroimaging Data-Sharing Initiative (INDI)Homo sapiensimaginglife sciencesmagnetic resonance imagingneuroimagingneuroscience This bucket contains multiple neuroimaging datasets that are part of the International Neuroimaging Data-Sharing Initiative. Raw human and non-human primate neuroimaging data include 1) Structural MRI; 2) Functional MRI; 3) Diffusion Tensor Imaging; 4) Electroencephalogram (EEG) In addition to the raw data, preprocessed data is also included for some datasets. A complete list of the available datasets can be seen in the documentation lonk provided below. Details → Usage examples
See 11 usage examples → NOAA Operational Forecast System (OFS)climatecoastaldisaster responseenvironmentalmeteorologicaloceanssustainabilitywaterweather ANNOUNCEMENTS: [NOS OFS Version Updates and Implementation of Upgraded Oceanographic Forecast Modeling Systems for Lakes Superior and Ontario; Effective October 25, 2022}(https://www.weather.gov/media/notification/pdf2/scn22-91_nos_loofs_lsofs_v3.pdf) For decades, mariners in the United States have depended on NOAA's Tide Tables for the best estimate of expected water levels. These tables provide accurate predictions of the astronomical tide (i.e., the change in water level due to the gravitational effects of the moon and sun and the rotation of the Earth); however, they cannot predict water-level changes due to wind, atmospheric pressure, and river flow, which are often significant. The National Ocean Service (NOS) has the mission and mandate to provide guidance and information to support navigation and coastal needs. To support this mission, NOS has been developing and implementing hydrodynamic model-based Operational Forecast Systems. This forecast guidance provides oceanographic information that helps mariners safely navigate their local waters. This national network of hydrodynamic models provides users with operational nowcast and forecast guidance (out to 48 – 120 hours) on parameters such as water levels, water temperature, salinity, and currents. These forecast systems are implemented in critical ports, harbors, estuaries, Great Lakes and coastal waters of the United States, and form a national backbone of real-time data, tidal predictions, data management and operational modeling. Nowcasts and forecasts are scientific predictions about the present and future states of water levels (and possibly currents and other relevant oceanographic variables, such as salinity and temperature) in a coastal area. These predictions rely on either observed data or forecasts from a numerical model. A nowcast incorporates recent (and often near real-time) observed meteorological, oceanographic, and/or river flow rate data. A nowcast covers the period from the recent past (e.g., the past few days) to the present, and it can make predictions for locations where observational data are not available. A forecast incorporates meteorological, oceanographic, and/or river flow rate forecasts and makes predictions for times where observational data will not be available. A forecast is usually initiated by the results of a nowcast. OFS generally runs four times per day (every 6 hours) on NOAA's Weather and Climate Operational Supercomputing Systems (WCOSS) in a standard Coastal Ocean Modeling Framework (COMF) developed by the Center for Operational Oceanographic Products and Services (CO-OPS). COMF is a set... Details → Usage examples
See 11 usage examples → Digital Earth Africa Sentinel-2 Level-2Aagriculturecogdeafricadisaster responseearth observationgeospatialnatural resourcesatellite imagerystacsustainability The Sentinel-2 mission is part of the European Union Copernicus programme for Earth observations. Sentinel-2 consists of twin satellites, Sentinel-2A (launched 23 June 2015) and Sentinel-2B (launched 7 March 2017). The two satellites have the same orbit, but 180° apart for optimal coverage and data delivery. Their combined data is used in the Digital Earth Africa Sentinel-2 product. Together, they cover all Earth’s land surfaces, large islands, inland and coastal waters every 3-5 days. Sentinel-2 data is tiered by level of pre-processing. Level-0, Level-1A and Level-1B data contain raw data fr... Details → Usage examples
See 10 usage examples → Department of Energy's Open Energy Data Initiative (OEDI)energyenvironmentalgeospatiallidarmodelsolarsustainability Data released under the Department of Energy's Open Energy Data Initiative (DOE). The Open Energy Data Initiative (OEDI) aims to improve and automate access of high-value energy data sets across the U.S. Department of Energy’s (DOE’s) programs, offices, and national laboratories. OEDI aims to make data actionable and discoverable by researchers and industry to accelerate analysis and advance innovation. Details → Usage examples
See 9 usage examples → Open NeuroDataarray tomographybiologyelectron microscopyimage processinglife scienceslight-sheet microscopymagnetic resonance imagingneuroimagingneuroscience This bucket contains multiple neuroimaging datasets (as Neuroglancer Precomputed Volumes) across multiple modalities and scales, ranging from nanoscale (electron microscopy), to microscale (cleared lightsheet microscopy and array tomography), and mesoscale (structural and functional magnetic resonance imaging). Additionally, many of the datasets include segmentations and meshes. Details → Usage examples
See 9 usage examples → DOE's Water Power Technology Office's (WPTO) US Wave datasetearth observationenergygeospatialmeteorologicalsustainabilitywater Released to the public as part of the Department of Energy's Open Energy Data Initiative, this is the highest resolution publicly available long-term wave hindcast dataset that – when complete – will cover the entire U.S. Exclusive Economic Zone (EEZ). Details → Usage examples
See 8 usage examples → NREL Wind Integration National Datasetenvironmentalgeospatialmeteorologicalsustainability Released to the public as part of the Department of Energy's Open Energy Data Initiative, the Wind Integration National Dataset (WIND) is an update and expansion of the Eastern Wind Integration Data Set and Western Wind Integration Data Set. It supports the next generation of wind integration studies. Details → Usage examples
See 8 usage examples → Toxicant Exposures and Responses by Genomic and Epigenomic Regulators of Transcription (TaRGET)bioinformaticsbiologyenvironmentalepigenomicsgeneticgenomiclife sciences The TaRGET (Toxicant Exposures and Responses by Genomic and Epigenomic Regulators of Transcription) Program is a research consortium funded by the National Institute of Environmental Health Sciences (NIEHS). The goal of the collaboration is to address the role of environmental exposures in disease pathogenesis as a function of epigenome perturbation, including understanding the environmental control of epigenetic mechanisms and assessing the utility of surrogate tissue analysis in mouse models of disease-relevant environmental exposures. Details → Usage examples
See 8 usage examples → USGS 3DEP LiDAR Point Cloudsagriculturedisaster responseelevationgeospatiallidarstacsustainability The goal of the USGS 3D Elevation Program (3DEP) is to collect elevation data in the form of light detection and ranging (LiDAR) data over the conterminous United States, Hawaii, and the U.S. territories, with data acquired over an 8-year period. This dataset provides two realizations of the 3DEP point cloud data. The first resource is a public access organization provided in Entwine Point Tiles format, which a lossless, full-density, streamable octree based on LASzip (LAZ) encoding. The second resource is a Requester Pays of the original, Raw LAZ (Compressed LAS) 1.4 3DEP format, and more co... Details → Usage examples
See 8 usage examples → World Bank - Light Every Nightcogdisaster responseearth observationsatellite imagerystac Light Every Night - World Bank Nightime Light Data – provides open access to all nightly imagery and data from the Visible Infrared Imaging Radiometer Suite Day-Night Band (VIIRS DNB) from 2012-2020 and the Defense Meteorological Satellite Program Operational Linescan System (DMSP-OLS) from 1992-2013. The underlying data are sourced from the NOAA National Centers for Environmental Information (NCEI) archive. Additional processing by the University of Michigan enables access in Cloud Optimized GeoTIFF format (COG) and search using the Spatial Temporal Asset Catalog (STAC) standard. The data is ... Details → Usage examples
See 8 usage examples → Clinical Proteomic Tumor Analysis Consortium 2 (CPTAC-2)cancergenomiclife sciencesSTRIDEStranscriptomics The Clinical Proteomic Tumor Analysis Consortium (CPTAC) is a national effort to accelerate the understanding of the molecular basis of cancer through the application of large-scale proteome and genome analysis, or proteogenomics. CPTAC-2 is the Phase II of the CPTAC Initiative (2011-2016). Datasets contain open RNA-Seq Gene Expression Quantification, miRNA-Seq Isoform Expression Quantification, and miRNA Expression Quantification data. Details → Usage examples
See 7 usage examples → Global Database of Events, Language and Tone (GDELT)disaster responseevents This project monitors the world's broadcast, print, and web news from nearly every corner of every country in over 100 languages and identifies the people, locations, organizations, counts, themes, sources, emotions, quotes, images and events driving our global society every second of every day. Details → Usage examples
See 7 usage examples → NOAA Joint Polar Satellite System (JPSS)agricultureclimatemeteorologicalsustainabilityweather Satellites in the JPSS constellation gather global measurements of atmospheric, terrestrial and oceanic conditions, including sea and land surface temperatures, vegetation, clouds, rainfall, snow and ice cover, fire locations and smoke plumes, atmospheric temperature, water vapor and ozone. JPSS delivers key observations for the Nation's essential products and services, including forecasting severe weather like hurricanes, tornadoes and blizzards days in advance, and assessing environmental hazards such as droughts, forest fires, poor air quality and harmful coastal waters. Further, JPSS w... Details → Usage examples
See 7 usage examples → ArcticDEMcogearth observationelevationgeospatialmappingopen source softwaresatellite imagerystac ArcticDEM - 2m GSD Digital Elevation Models (DEMs) and mosaics from 2007 to the present. The ArticDEM project seeks to fill the need for high-resolution time-series elevation data in the Arctic. The time-dependent nature of the strip DEM files allows users to perform change detection analysis and to compare observations of topography data acquired in different seasons or years. The mosaic DEM tiles are assembled from multiple strip DEMs with the intention of providing a more consistent and comprehensive product over large areas. ArcticDEM data is constructed from in-track and cross-track high-... Details → Usage examples
See 6 usage examples → BossDB Open Neuroimagery Datasetscalcium imagingelectron microscopyimaginglife scienceslight-sheet microscopymagnetic resonance imagingneuroimagingneurosciencevolumetric imagingx-rayx-ray microtomographyx-ray tomography This data ecosystem, Brain Observatory Storage Service & Database (BossDB), contains several neuro-imaging datasets across multiple modalities and scales, ranging from nanoscale (electron microscopy), to microscale (cleared lightsheet microscopy and array tomography), and mesoscale (structural and functional magnetic resonance imaging). Additionally, many of the datasets include dense segmentation and meshes. Details → Usage examples
See 6 usage examples → Low Altitude Disaster Imagery (LADI) Datasetaerial imagerycoastalcomputer visiondisaster responseearth observationearthquakesgeospatialimage processingimaginginfrastructurelandmachine learningmappingnatural resourceseismologytransportationurbanwater The Low Altitude Disaster Imagery (LADI) Dataset consists of human and machine annotated airborne images collected by the Civil Air Patrol in support of various disaster responses from 2015-2019. The initial release of LADI focuses on the Atlantic hurricane seasons and coastal states along the Atlantic Ocean and Gulf of Mexico. Annotations are included for major hurricanes of Harvey, Maria, and Florence. Two key distinctions are the low altitude, oblique perspective of the imagery and disaster-related features, which are rarely featured in computer vision benchmarks and datasets. Details → Usage examples
See 6 usage examples → NOAA Rapid Refresh Forecast System (RRFS) [Prototype]agricultureclimatemeteorologicalsustainabilityweather The Rapid Refresh Forecast System (RRFS) is the National Oceanic and Atmospheric Administration’s (NOAA) next generation convection-allowing, rapidly-updated ensemble prediction system, currently scheduled for operational implementation in 2024. The operational configuration will feature a 3 km grid covering North America and include deterministic forecasts every hour out to 18 hours, with deterministic and ensemble forecasts to 60 hours four times per day at 00, 06, 12, and 18 UTC.The RRFS will provide guidance to support forecast interests including, but not limited to, aviation, severe convective weather, renewable energy, heavy precipitation, and winter weather on timescales where rapidly-updated guidance is particularly useful. The RRFS is underpinned by the Unified Forecast System (UFS), a community-based Earth modeling initiative, and benefits from collaborative development efforts across NOAA, academia, and research institutions. This bucket provides access to real time, experimental RRFS prototype output as of October 2022. This bucket also holds output from past experimental RRFS prototypes that were evaluated as a part of NOAA testbed projects. The immediate section describes the data for the real time system. The section that follows thereafter describes outputs from three past NOAA Testbed experiments. Real time, experimental RRFS Prototype output The real-time RRFS prototype is experimental and evolving. It is not under 24x7 monitoring and is not operational. Output may be delayed or missing. Outputs will change. When significant changes to output take place, this description will be updated. We currently provide hourly deterministic forecasts at 3 km grid spacing over the CONUS out to 60 hours at 00 and 12 UTC, and out to 18 hours at other times. Future enhancements will include an ensemble forecast component and expansion to the planned North American domain. All forecasts are initialized from a hybrid 3DEnVar data assimilation system with
hourly updates.Output is available on the S3 bucket for every third cycle, and is organized by cycle day and time of day. For example, rrfs.t00z.natlev.f018.conus_3km.grib2 Meaning that this is the RRFS_A initialized at 00 UTC, covers the CONUS domain, and is the native level post-processed gridded data at hour 18. This output is on a Lambert Conic Conformal domain at 3 km grid spacing. The second output file in grib2 format is: rrfs.t00z.prslev.f018.conus_3km.grib2 Meaning that this is the pressure level post-processed gridded data. Past output from NOAA Testbed Experiments This bucket also provides datasets from three of the 2021 NOAA Testbed Experiments. During each of these experiments, a prototype version of RRFS under development was run. The following is a high-level overview dates and RRFS configurations for each of the Testbed Experiments. 2021 Hazardous Weather Testbed (HWT) Spring Forecast Experiment (May 3 through June 4 2021) and 2021 Hydrometeorological Testbed Annual Flash Flood and Intense Rainfall Experiment (FFaIR) (June 21 through July 23 2021, excluding the week of July 4). A 9-member multi-physics ensemble with stochastic perturbations run once per day at 3 km grid spacing covering North America out to 60 hours. Initial conditions and lateral boundary conditions are taken from the GFS and GEFS. 2021-2022 Hydrometeorological Testbed Winter Weather Experiment (WWE) (mid November through mid-March). Select cases only. Deterministic forecasts were run once per day at 00 UTC at 3 km grid spacing covering the CONUS out to 60 hours. A 36-member, 3 km ensemble Kalman filter data assimilation approach is implemented through hourly cycling starting at 18 UTC on the previous day. For each cycle of the HWT and FFaIR experiments, the dataset
is organized by cycle day, time of day, and member. For example, rrfs.t00z.mem01.naf024.grib2 Meaning that this is RRFS ensemble member 1 initialized at 00 UTC, covers the North American domain, and is the post-processed gridded data at hour 24. This output is on a rotated latitude-longitude domain at 3 km grid spacing. These are large files and users may wish to subset or re-project the grid after downloading. We recommend using the WGRIB2 application for such purposes. The second output file in grib2 format is as follows: rrfs.t00z.mem01.testbed.conusf020.grib2 These grids have been subset from the much larger North American domain to a CONUS domain on a Lambert Conic Conformal projection and also contain significantly fewer fields, resulting in smaller files. Graphics for select runs are also included in a plots/ directory under each experiment day for quick, yet simple visualization. For each cycle of the WWE, the dataset is organized by cycle day and time of day. For example, BGDAWP.GrbF12 Meaning that this is the forecast initialized at 00 UTC, covers the CONUS domain, and is the pressure level post-processed gridded data at forecast hour 18. This output is on a Lambert Conic Conformal grid at 3 km grid spacing. The second output file in grib2 format is as follows: testbed.conusf030.grib2 These grids contain significantly fewer fields, resulting in smaller files. This work is supported by the Unified Forecast System Research to Operation (UFS R2O) Project which is jointly funded by NOAA’s Office of Science and Technology Integration (OSTI) of National Weather Service (NWS) and Weather Program Office (WPO), [Joint Technology Transfer Initiative (JTTI)] of the Office of Oceanic and Atmospheric Research (OAR). DISCLAIMER The o... Details → Usage examples
See 6 usage examples → Open Bioinformatics Reference Data for Galaxybioinformaticsbiologygeneticgenomiclife sciencesreference index This dataset provides genomic reference data and software packages for use with Galaxy and Bioconductor applications. The reference data is available for hundreds of reference genomes and has been formatted for use with a variety of tools. The available configuration files make this data easily incorporable with a local Galaxy server without additional data preparation. Additionally, Bioconductor's AnnotationHub and ExperimentHub data are provided for use via R packag... Details → Usage examples
See 6 usage examples → PoroTomogeospatialgeothermalimage processingseismology Released to the public as part of the Department of Energy's Open Energy Data Initiative, these data represent vertical and horizontal distributed acoustic sensing (DAS) data collected as part of the Poroelastic Tomography (PoroTomo) project funded in part by the Office of Energy Efficiency and Renewable Energy (EERE), U.S. Department of Energy. Details → Usage examples
See 6 usage examples → Reference Elevation Model of Antarctica (REMA)cogearth observationelevationgeospatialmappingopen source softwaresatellite imagerystac The Reference Elevation Model of Antarctica - 2m GSD Digital Elevation Models (DEMs) and mosaics from 2009 to the present. The REMA project seeks to fill the need for high-resolution time-series elevation data in the Antarctic. The time-dependent nature of the strip DEM files allows users to perform change detection analysis and to compare observations of topography data acquired in different seasons or years. The mosaic DEM tiles are assembled from multiple strip DEMs with the intention of providing a more consistent and comprehensive product over large areas. REMA data is constructed from in... Details → Usage examples
See 6 usage examples → CAM6 Data Assimilation Research Testbed (DART) Reanalysis: Cloud-Optimized Datasetatmosphereclimateclimate modeldata assimilationforecastgeosciencegeospatiallandmeteorologicalweatherzarr This is a cloud-hosted subset of the CAM6+DART (Community Atmosphere Model version 6 Data Assimilation Research Testbed) Reanalysis dataset. These data products are designed to facilitate a broad variety of research using the NCAR CESM 2.1 (National Center for Atmospheric Research's Community Earth System Model version 2.1), including model evaluation, ensemble hindcasting, data assimilation experiments, and sensitivity studies. They come from an 80 member ensemble reanalysis of the global troposphere and stratosphere using DART and CAM6. The data products represent states of the atmospher... Details → Usage examples
See 5 usage examples → CoMMpass from the Multiple Myeloma Research FoundationcancergeneticgenomicSTRIDESwhole genome sequencing The Relating Clinical Outcomes in Multiple Myeloma to Personal Assessment of Genetic Profile study is the Multiple Myeloma Research Foundation (MMRF)’s landmark personalized medicine initiative. CoMMpass is a longitudinal observation study of around 1000 newly diagnosed myeloma patients receiving various standard approved treatments. The MMRF’s vision is to track the treatment and results for each CoMMpass patient so that someday the information can be used to guide decisions for newly diagnosed patients. CoMMpass checked on patients every 6 months for 8 years, collecting tissue samples, gene... Details → Usage examples
See 5 usage examples →
Community Earth System Model Large Ensemble (CESM LENS)atmosphereclimateclimate modelgeospatialicelandmodeloceanssustainabilityzarr The Community Earth System Model (CESM) Large Ensemble Numerical Simulation (LENS) dataset includes a 40-member ensemble of climate simulations for the period 1920-2100 using historical data (1920-2005) or assuming the RCP8.5 greenhouse gas concentration scenario (2006-2100), as well as longer control runs based on pre-industrial conditions. The data comprise both surface (2D) and volumetric (3D) variables in the atmosphere, ocean, land, and ice domains. The total data volume of the original dataset is ~500TB, which has traditionally been stored as ~150,000 individual CF/NetCDF files on disk o... Details → Usage examples
See 5 usage examples → First Street Foundation (FSF) Flood Risk Summary Statisticsagricultureclimatemodelstatisticssustainabilitywaterweather CSV files of flood statistics for the 48 contiguous states at the congressional district, county, and zip code level. The CSV for each of these geographical extents includes statistics on the amount of properties at risk according to FEMA, the number of properties at risk according to First Street Foundation, and the difference between the two. Details → Usage examples
See 5 usage examples → Global Seasonal Sentinel-1 Interferometric Coherence and Backscatter Data Setagriculturecogearth observationearthquakesecosystemsenvironmentalgeologygeophysicsgeospatialglobalinfrastructuremappingnatural resourcesatellite imagerysynthetic aperture radarurban This data set is the first-of-its-kind spatial representation of multi-seasonal, global SAR repeat-pass interferometric coherence and backscatter signatures. Global coverage comprises all land masses and ice sheets from 82 degrees northern to 79 degress southern latitude. The data set is derived from high-resolution multi-temporal repeat-pass interferometric processing of about 205,000 Sentinel-1 Single-Look-Complex data acquired in Interferometric Wide-Swath mode (Sentinel-1 IW mode) from 1-Dec-2019 to 30-Nov-2020. The data set was developed by Earth Big Data LLC and Gamma Remote Sensing AG, under contract for NASA's Jet Propulsion Laboratory. ... Details → Usage examples
See 5 usage examples → NOAA National Water Model CONUS Retrospective Datasetagricultureagricultureclimatedisaster responseenvironmentalsustainabilitytransportationweather The NOAA National Water Model Retrospective dataset contains input and output from multi-decade CONUS retrospective simulations. These simulations used meteorological input fields from meteorological retrospective datasets. The output frequency and fields available in this historical NWM dataset differ from those contained in the real-time operational NWM forecast model. One application of this dataset is to provide historical context to current near real-time streamflow, soil moisture and snowpack conditions. The retrospective data can be used to infer flow frequencies and perform temporal analyses with hourly streamflow output and 3-hourly land surface output. This dataset can also be used in the development of end user applications which require a long baseline of data for system training or verification purposes. Currently there are three versions of the NWM retrospective dataset A 42-year (February 1979 through December 2020) retrospective simulation using version 2.1 of the National Water Model. A 26-year (January 1993 through December 2018) retrospective simulation using version 2.0 of the National Water Model. A 25-year (January 1993 through December 2017) retrospective simulation using version 1.2 of the National Water Model. Version 2.1 uses forcings from the Office of Water Prediction Analysis of Record for Calibration (AORC) dataset while Version 2.0 and version 1.2 use input meteorological forcing from the North American Land Data Assimilation (NLDAS) data set. Note that no streamflow or other data assimilation is performed within any of the NWM retrospective simulations. NWM Retrospective data is available in two formats, NetCDF and Zarr. The NetCDF files contain the full set of NWM output data, while the Zarr files contain a subset of NWM output fields that vary with model version. NWM V2.1: All model output and forcing input fields are available in the NetCDF format. All model output fields along with the precipitation forcing field are available in the Zarr format. NWM V2.0: All model output fields are available in NetCDF format. Model channel output including streamflow and related fields are available in Zarr format. NWM V1.2: All model output fields are available in NetCDF format. A table listing the data available within each NetCDF and Zarr file is located in the 'documentation page'. This data includes meteorologic... Details → Usage examples
See 5 usage examples → The Human Connectome Projectbiologyimaginglife sciencesneurobiologyneuroimagingneuroscience The Human Connectome Project (HCP Young Adult, HCP-YA) is mapping the healthy human connectome by collecting and freely distributing neuroimaging and behavioral data on 1,200 normal young adults, aged 22-35. Details → Usage examples
See 5 usage examples → Basic Local Alignment Sequences Tool (BLAST) Databasesbioinformaticsbiologygeneticgenomichealthlife sciencesproteinreference indextranscriptomics A centralized repository of pre-formatted BLAST databases created by the National Center for Biotechnology Information (NCBI). Details → Usage examples
See 4 usage examples → Boreas Autonomous Driving Datasetautonomous vehiclescomputer visionlidarrobotics This autonomous driving dataset includes data from a 128-beam Velodyne Alpha-Prime lidar, a 5MP Blackfly camera, a 360-degree Navtech radar, and post-processed Applanix POS LV GNSS data. This dataset was collect in various weather conditions (sun, rain, snow) over the course of a year. The intended purpose of this dataset is to enable benchmarking of long-term all-weather odometry and metric localization across various sensor types. In the future, we hope to also support an object detection benchmark. Details → Usage examples
See 4 usage examples → JMA Himawari-8agriculturedisaster responseearth observationgeospatialmeteorologicalsatellite imagerysustainabilityweather Himawari-8, stationed at 140E, owned and operated by the Japan Meteorological Agency (JMA), is a geostationary meteorological satellite, with Himawari-9 as on-orbit back-up, that provides constant and uniform coverage of east Asia, and the west and central Pacific regions from around 35,800 km above the equator with an orbit corresponding to the period of the earth’s rotation. This allows JMA weather offices to perform uninterrupted observation of environmental phenomena such as typhoons, volcanoes, and general weather systems. Archive data back to July 2015 is available for Full Disk (AHI-L1... Details → Usage examples
See 4 usage examples → Maxar Open Data Programcogdisaster responseearth observationgeospatialsatellite imagerystacsustainability Pre and post event high-resolution satellite imagery in support of emergency planning, risk assessment, monitoring of staging areas and emergency response, damage assessment, and recovery. These images are generated using the Maxar ARD pipeline, tiled on an organized grid in analysis-ready cloud-optimized formats. Details → Usage examples
See 7 usage examples → Mouse Brain Anatomy: MouseLight Imagerybiologyfluorescence imagingimage processingimaginglife sciencesmicroscopyneurobiologyneuroimagingneuroscience This data set, made available by Janelia's MouseLight project, consists of images and neuron annotations of the Mus musculus brain, stored in formats suitable for viewing and annotation using the HortaCloud cloud-based annotation system. Details → Usage examples
See 4 usage examples → NAIP on AWSaerial imageryagriculturecogearth observationgeospatialnatural resourceregulatorysustainability The National Agriculture Imagery Program (NAIP) acquires aerial imagery during the agricultural growing seasons in the continental U.S. This "leaf-on" imagery andtypically ranges from 60 centimeters to 100 centimeters in resolution and is available from the naip-analytic Amazon S3 bucket as 4-band (RGB + NIR) imagery in MRF format, on naip-source Amazon S3 bucket as 4-band (RGB + NIR) in uncompressed Raw GeoTiff format and naip-visualization as 3-band (RGB) Cloud Optimized GeoTiff format. NAIP data is delivered at the state level; every year, a number of states receive updates, with ... Details → Usage examples
See 4 usage examples → NREL National Solar Radiation Databaseearth observationenergygeospatialmeteorologicalsolarsustainability Released to the public as part of the Department of Energy's Open Energy Data Initiative, the National Solar Radiation Database (NSRDB) is a serially complete collection of hourly and half-hourly values of the three most common measurements of solar radiation – global horizontal, direct normal, and diffuse horizontal irradiance — and meteorological data. These data have been collected at a sufficient number of locations and temporal and spatial scales to accurately represent regional solar radiation climates. Details → Usage examples
See 4 usage examples → OpenCell on AWSbiologycell biologycell imagingcomputer visionfluorescence imagingimaginglife sciencesmachine learningmicroscopy The OpenCell project is a proteome-scale effort to measure the localization and interactions of human proteins using high-throughput genome engineering to endogenously tag thousands of proteins in the human proteome. This dataset consists of the raw confocal fluorescence microscopy images for all tagged cell lines in the OpenCell library. These images can be interpreted both individually, to determine the localization of particular proteins of interest, and in aggregate, by training machine learning models to classify or quantify subcellular localization patterns. Details → Usage examples
See 4 usage examples → Sea Surface Temperature Daily Analysis: European Space Agency Climate Change Initiative product version 2.1climateearth observationenvironmentalgeospatialglobaloceans Global daily-mean sea surface temperatures, presented on a 0.05° latitude-longitude grid, with gaps between available daily observations filled by statistical means, spanning late 1981 to recent time. Suitable for large-scale oceanographic meteorological and climatological applications, such as evaluating or constraining environmental models or case-studies of marine heat wave events. Includes temperature uncertainty information and auxiliary information about land-sea fraction and sea-ice coverage. For reference and citation see: www.nature.com/articles/s41597-019-0236-x. Details → Usage examples
See 4 usage examples → Virginia Coastal Resilience Master Plan, Phase 1 - December 2021coastalfloods The Virginia Coastal Resilience Master Plan builds on the 2020 Virginia Coastal Resilience Master Planning Framework, which outlined the goals and principles of the Commonwealth’s statewide coastal resilience strategy. Recognizing the urgent challenge flooding already poses, the Commonwealth developed Phase One of the Master Plan on an accelerated timeline and focused this first assessment on the impacts of tidal and storm surge coastal flooding on coastal Virginia. The Master Plan leveraged the combined efforts of more than two thousand stakeholders, subject matter experts, and government personnel. We centered the development of this plan around three core components: A Technical Study compiled essential data, research, processes, products, and resilience efforts in the Coastal Resilience Database, which forms much of basis of this plan and the Coastal Resilience Web Explorer; A Technical Advisory Committee supported coordination across key stakeholders and ensured the incorporation of the best available subject matter knowledge, data, and methods into this plan; and Stakeholder Engagement captured diverse resilience perspectives from residents, local and regional officials, and other stakeholders across Virginia’s coastal communities to drive regionally specific resilience priorities.Data products used and generated for the Virginia Coastal Resilience. This dataset represents the data that was developed for the technical study. Appendix F - Data Product List provides a list of available data. Other Appendix documents provide the inpu... Details → Usage examples
See 4 usage examples → Yale-CMU-Berkeley (YCB) Object and Model Setrobotics This project primarily aims to facilitate performance benchmarking in robotics research. The dataset provides mesh models, RGB, RGB-D and point cloud images of over 80 objects. The physical objects are also available via the YCB benchmarking project. The data are collected by two state of the art systems: UC Berkley's scanning rig and the Google scanner. The UC Berkley's scanning rig data provide meshes generated with Poisson reconstruction, meshes generated with volumetric range image integration, textured versions of both meshes, Kinbody files for using the meshes with OpenRAVE, 600 ... Details → Usage examples
See 4 usage examples → iSDAsoilagricultureanalyticsbiodiversityconservationdeep learningfood securitygeospatialmachine learningsatellite imagery iSDAsoil is a resource containing soil property predictions for the entire African continent, generated using machine learning. Maps for over 20 different soil properties have been created at 2 different depths (0-20 and 20-50cm). Soil property predictions were made using machine learning coupled with remote sensing data and a training set of over 100,000 analyzed soil samples. Included in this datset are images of predicted soil properties, model error and satellite covariates used in the mapping process. Details → Usage examples
See 4 usage examples → Beat Acute Myeloid Leukemia (AML) 1.0cancergeneticgenomicHomo sapienslife sciencesSTRIDES Beat AML 1.0 is a collaborative research program involving 11 academic medical centers who worked collectively to better understand drugs and drug combinations that should be prioritized for further development within clinical and/or molecular subsets of acute myeloid leukemia (AML) patients. Beat AML 1.0 provides the largest-to-date dataset on primary acute myeloid leukemia samples offering genomic, clinical, and drug response.This dataset contains open Clinical Supplement and RNA-Seq Gene Expression Quantification data.This dataset also contains controlled Whole Exome Sequencing (WXS) and R... Details → Usage examples
See 3 usage examples → Cell Organelle Segmentation in Electron Microscopy (COSEM) on AWScell biologycomputer visionelectron microscopyimaginglife sciencesorganelle High resolution images of subcellular structures. Details → Usage examples
See 3 usage examples → Clinical Trial Sequencing Project - Diffuse Large B-Cell Lymphomacancergenomiclife sciencesSTRIDEStranscriptomicswhole genome sequencing The goal of the project is to identify recurrent genetic alterations (mutations, deletions, amplifications, rearrangements) and/or gene expression signatures. National Cancer Institute (NCI) utilized whole genome sequencing and/or whole exome sequencing in conjunction with transcriptome sequencing. The samples were processed and submitted for genomic characterization using pipelines and procedures established within The Cancer Genome Analysis (TCGA) project. Details → Usage examples
See 3 usage examples → Finnish Meteorological Institute Weather Radar Dataagricultureearth observationmeteorologicalsustainabilityweather The up-to-date weather radar from the FMI radar network is available as Open Data. The data contain both single radar data along with composites over Finland in GeoTIFF and HDF5-formats. Available composite parameters consist of radar reflectivity (DBZ), rainfall intensity (RR), and precipitation accumulation of 1, 12, and 24 hours. Single radar parameters consist of radar reflectivity (DBZ), radial velocity (VRAD), rain classification (HCLASS), and Cloud top height (ETOP 20). Raw volume data from singe radars are also provided in HDF5 format with ODIM 2.3 conventions. Radar data becomes avail... Details → Usage examples
See 3 usage examples → Foundation Medicine Adult Cancer Clinical Dataset (FM-AD)cancergenomic The Foundation Medicine Adult Cancer Clinical Dataset (FM-AD) is a study conducted by Foundation Medicine Inc (FMI). Genomic profiling data for approximately 18,000 adult patients with a diverse array of cancers was generated using FoundationeOne, FMI's commercially available, comprehensive genomic profiling assay. This dataset contains open Clinical and Biospecimen data. Details → Usage examples
See 3 usage examples → MIMIC-III (‘Medical Information Mart for Intensive Care’)bioinformaticshealthlife sciencesnatural language processingus MIMIC-III (‘Medical Information Mart for Intensive Care’) is a large, single-center database comprising information relating to patients admitted to critical care units at a large tertiary care hospital. Data includes vital signs, medications, laboratory measurements, observations and notes charted by care providers, fluid balance, procedure codes, diagnostic codes, imaging reports, hospital length of stay, survival data, and more. The database supports applications including academic and industrial research, quality improvement initiatives, and higher education coursework. The MIMIC-I... Details → Usage examples
See 3 usage examples → Medical Segmentation Decathloncomputed tomographyhealthimaginglife sciencesmagnetic resonance imagingmedicineniftisegmentation With recent advances in machine learning, semantic segmentation algorithms are becoming increasingly general purpose and translatable to unseen tasks. Many key algorithmic advances in the field of medical imaging are commonly validated on a small number of tasks, limiting our understanding of the generalisability of the proposed contributions. A model which works out-of-the-box on many tasks, in the spirit of AutoML, would have a tremendous impact on healthcare. The field of medical imaging is also missing a fully open source and comprehensive benchmark for general purpose algorithmic validati... Details → Usage examples
See 3 usage examples → Multiview Extended Video with Activities (MEVA)computer visionurbanusvideo The Multiview Extended Video with Activities (MEVA) dataset consists video data of human activity, both scripted and unscripted, collected with roughly 100 actors over several weeks. The data was collected with 29 cameras with overlapping and non-overlapping fields of view. The current release consists of about 328 hours (516GB, 4259 clips) of video data, as well as 4.6 hours (26GB) of UAV data. Other data includes GPS tracks of actors, camera models, and a site map. We have also released annotations for roughly 184 hours of data. Further updates are planned. Details → Usage examples
See 3 usage examples → OpenAlex datasetgraphjsonmetadatascholarly communication An open, comprehensive index of scolarly papers, citations, authors, institutions, and journals. Details → Usage examples
See 3 usage examples → The Human Microbiome Projectamino acidfastafastqgeneticgenomiclife sciencesmetagenomicsmicrobiome The NIH-funded Human Microbiome Project (HMP) is a collaborative effort of over 300 scientists from more than 80 organizations to comprehensively characterize the microbial communities inhabiting the human body and elucidate their role in human health and disease. To accomplish this task, microbial community samples were isolated from a cohort of 300 healthy adult human subjects at 18 specific sites within five regions of the body (oral cavity, airways, urogenital track, skin, and gut). Targeted sequencing of the 16S bacterial marker gene and/or whole metagenome shotgun sequencing was performe... Details → Usage examples
See 3 usage examples → 4D Nucleome (4DN)bioinformaticsbiologygeneticgenomicimaginglife sciences The goal of the National Institutes of Health (NIH) Common Fund’s 4D Nucleome (4DN) program is to study the three-dimensional organization of the nucleus in space and time (the 4th dimension). The nucleus of a cell contains DNA, the genetic “blueprint” that encodes all of the genes a living organism uses to produce proteins needed to carry out life-sustaining cellular functions. Understanding the conformation of the nuclear DNA and how it is maintained or changes in response to environmental and cellular cues over time will provide insights into basic biology as well as aspects of human health... Details → Usage examples
See 2 usage examples → Atmospheric Models from Météo-Franceagricultureclimatedisaster responseearth observationenvironmentalmeteorologicalmodelweather Global and high-resolution regional atmospheric models from Météo-France.
Dozens of atmospheric variables are avail... Details → Usage examples
See 2 usage examples → Cancer Genome Characterization Initiatives - Burkitt Lymphoma, HIV+ Cervical Cancercancergenomiclife sciencesSTRIDEStranscriptomics The Cancer Genome Characterization Initiatives (CGCI) program supports cutting-edge genomics research of adult and pediatric cancers. CGCI investigators develop and apply advanced sequencing methods that examine genomes, exomes, and transcriptomes within various types of tumors. The program includes Burkitt Lymphoma Genome Sequencing Project (BLGSP) project and HIV+ Tumor Molecular Characterization Project - Cervical Cancer (HTMCP-CC) project. The dataset contains open Clinical Supplement, Biospecimen Supplement, RNA-Seq Gene Expression Quantification, miRNA-Seq Isoform Expression Quantificati... Details → Usage examples
See 2 usage examples → Copernicus Digital Elevation Model (DEM)agriculturecogdisaster responseearth observationelevationgeospatialsatellite imagerysustainability The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, infrastructure and vegetation. We provide two instances of Copernicus DEM named GLO-30 Public and GLO-90. GLO-90 provides worldwide coverage at 90 meters. GLO-30 Public provides limited worldwide coverage at 30 meters because a small subset of tiles covering specific countries are not yet released to the public by the Copernicus Programme. Note that in both cases ocean areas do not have tiles, there one can assume height values equal to zero. Data is provided as Cloud Optimized Ge... Details → Usage examples
See 2 usage examples → DNAStack COVID19 SRA DatabambioinformaticscoronavirusCOVID-19fastafastqgeneticgenomicglobalhealthlife scienceslong read sequencingSARS-CoV-2vcfviruswhole genome sequencing The Sequence Read Archive (SRA) is the primary archive of high-throughput sequencing data, hosted by the National Institutes of Health (NIH). The SRA represents the largest publicly available repository of SARS-CoV-2 sequencing data. This dataset was created by DNAstack using SARS-CoV-2 sequencing data sourced from the SRA. Where possible, raw sequence data were processed by DNAstack through a unified bioinformatics pipeline to produce genome assemblies and variant calls. The use of a standardized workflow to produce this harmonized dataset allows public data generated using different methodol... Details → Usage examples
See 2 usage examples → DigitalCorporacomputer forensicscomputer securityCSIcyber securitydigital forensicsimage processingimaginginformation retrievalinternetintrusion detectionmachine learningmachine translationtext analysis Disk images, memory dumps, network packet captures, and files for use in digital forensics research and education. All of this information is accessible through the digitalcorpora.org website, and made available at s3://digitalcorpora/. Some of these datasets implement scenarios that were performed by students, faculty, and others acting in persona. As such, the information is synthetic and may be used without prior authorization or IRB approval. Details of these datasets can be found at Details → Usage examples
See 2 usage examples → Hecatomb Databasesbioinformaticsgeneticgenomiclife sciencesmetagenomicsviruswhole genome sequencing Preprocessed databases for use with the Hecatomb pipeline for viral and phage sequence annotation. Details → Usage examples
See 2 usage examples → NOAA Climate Forecast System (CFS)agricultureclimatemeteorologicalsustainabilityweather The Climate Forecast System (CFS) is a model representing the global interaction between Earth's oceans, land, and atmosphere. Produced by several dozen scientists under guidance from the National Centers for Environmental Prediction (NCEP), this model offers hourly data with a horizontal resolution down to one-half of a degree (approximately 56 km) around Earth for many variables. CFS uses the latest scientific approaches for taking in, or assimilating, observations from data sources including surface observations, upper air balloon observations, aircraft observations, and satellite obser... Details → Usage examples
See 2 usage examples → NOAA Emergency Response Imageryaerial imageryclimatecogdisaster responsesustainabilityweather In order to support NOAA's homeland security and emergency response requirements, the National Geodetic Survey Remote Sensing Division (NGS/RSD) has the capability to acquire and rapidly disseminate a variety of spatially-referenced datasets to federal, state, and local government agencies, as well as the general public. Remote sensing technologies used for these projects have included lidar, high-resolution digital cameras, a film-based RC-30 aerial camera system, and hyperspectral imagers. Examples of rapid response initiatives include acquiring high resolution images with the Emerge/App... Details → Usage examples
See 2 usage examples → NOAA World Ocean Database (WOD)climateoceanssustainability The World Ocean Database (WOD) is the largest uniformly formatted, quality-controlled, publicly available historical subsurface ocean profile database. From Captain Cook's second voyage in 1772 to today's automated Argo floats, global aggregation of ocean variable information including temperature, salinity, oxygen, nutrients, and others vs. depth allow for study and understanding of the changing physical, chemical, and to some extent biological state of the World's Oceans. Browse the bucket via the AWS S3 explorer: https://noaa-wod-pds.s3.amazonaws.com/index.html Details → Usage examples
See 2 usage examples → Pancreatic Cancer Organoid ProfilingcancergeneticgenomicSTRIDEStranscriptomicswhole genome sequencing This study generated a collection of patient-derived pancreatic normal and cancer organoids and it was sequenced using Whole Genome Sequencing (WGS), Whole Exome Sequencing (WXS) and RNA-Seq as well as matched tumor and normal tissue if available. The study provides a valuable resource for pancreatic cancer researchers. The dataset contains open RNA-Seq Gene Expression Quantification data and controlled WGS/WXS/RNA-Seq Aligned Reads, WXS Annotated Somatic Mutation, WXS Raw Somatic Mutation, and RNA-Seq Splice Junction Quantification. Details → Usage examples
See 2 usage examples → Protein Data Bank 3D Structural Biology Dataamino acidarchivesbioinformaticsbiomolecular modelingcell biologychemical biologyCOVID-19electron microscopyelectron tomographyenzymelife sciencesmoleculenuclear magnetic resonancepharmaceuticalproteinprotein templateSARS-CoV-2structural biologyx-ray crystallography The "Protein Data Bank (PDB) archive" was established in 1971 as the first open-access digital data archive in biology. It is a collection of three-dimensional (3D) atomic-level structures of biological macromolecules (i.e., proteins, DNA, and RNA) and their complexes with one another and various small-molecule ligands (e.g., US FDA approved drugs, enzyme co-factors). For each PDB entry (unique identifier: 1abc or PDB_0000001abc) multiple data files contain information about the 3D atomic coordinates, sequences of biological macromolecules, information about any small molecules/ligan... Details → Usage examples
See 2 usage examples → RAPID NRT Flood Mapsagriculturedisaster responseearth observationenvironmentalwater Near Real-time and archival data of High-resolution (10 m) flood inundation dataset over the Contiguous United States, developed based on the Sentinel-1 SAR imagery (2016-current) archive, using an automated Radar Produced Inundation Diary (RAPID) algorithm. Details → Usage examples
See 2 usage examples → STOIC2021 Trainingcomputed tomographycomputer visioncoronavirusCOVID-19grand-challenge.orgimaginglife sciencesSARS-CoV-2 The STOIC project collected Computed Tomography (CT) images of 10,735 individuals suspected of being infected with SARS-COV-2 during the first wave of the pandemic in France, from March to April 2020. For each patient in the training set, the dataset contains binary labels for COVID-19 presence, based on RT-PCR test results, and COVID-19 severity, defined as intubation or death within one month from the acquisition of the CT scan. This S3 bucket contains the training sample of the STOIC dataset as used in the STOIC2021 challenge on grand-challenge.org. Details → Usage examples
See 2 usage examples → Sentinel-1 SLC dataset for South and Southeast Asia, Taiwan, Korea and Japandisaster responseearth observationenvironmentalgeospatialsatellite imagerysustainabilitysynthetic aperture radar The S1 Single Look Complex (SLC) dataset contains Synthetic Aperture Radar (SAR) data in the C-Band wavelength. The SAR sensors are installed on a two-satellite (Sentinel-1A and Sentinel-1B) constellation orbiting the Earth with a combined revisit time of six days, operated by the European Space Agency. The S1 SLC data are a Level-1 product that collects radar amplitude and phase information in all-weather, day or night conditions, which is ideal for studying natural hazards and emergency response, land applications, oil spill monitoring, sea-ice conditions, and associated climate change effec... Details → Usage examples
See 2 usage examples → Terra Fusion Data Samplergeospatialsatellite imagerysustainability The Terra Basic Fusion dataset is a fused dataset of the original Level 1 radiances from the five Terra instruments. They have been fully validate to contain the original Terra instrument Level 1 data. Each Level 1 Terra Basic Fusion file contains one full Terra orbit of data and is typically 15 – 40 GB in size, depending on how much data was collected for that orbit. It contains instrument radiance in physical units; radiance quality indicator; geolocation for each IFOV at its native resolution; sun-view geometry; bservation time; and other attributes/metadata. It is stored in HDF5, conformed to CF conventions, and accessible by netCDF-4 enhanced models. It’s naming convention follows: TERRA_BF_L1B_OXXXX_YYYYMMDDHHMMSS_F000_V000.h5. A concise description of the dataset, along with links to complete documentation and available software tools, can be found on the Terra Fusion project page: https://terrafusion.web.illinois.edu. Terra is the flagship satellite of NASA’s Earth Observing System (EOS). It was launched into orbit on December 18, 1999 and carries five instruments. These are the Moderate-resolution Imaging Spectroradiometer (MODIS), the Multi-angle Imaging SpectroRadiometer (MISR), the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER), the Clouds and Earth’s Radiant Energy System (CERES), and the Measurements of Pollution in the Troposphere (MOPITT). The Terra Basic Fusion dataset is an easy-to-access record of the Level 1 radiances for instruments on... Details → Usage examples
See 2 usage examples → 3DCoMPaT: Composition of Materials on Parts of 3D Thingscomputer visionmachine learning 3D CoMPaT is a richly annotated large-scale dataset of rendered compositions of Materials on Parts of thousands of unique 3D Models. This dataset primarily focuses on stylizing 3D shapes at part-level with compatible materials. Each object with the applied part-material compositions is rendered from four equally spaced views as well as four randomized views. We introduce a new task, called Grounded CoMPaT Recognition (GCR), to collectively recognize and ground compositions of materials on parts of 3D objects. We present two variations of this task and adapt state-of-art 2D/3D deep learning met... Details → Usage examples
See 1 usage example → A2D2: Audi Autonomous Driving Datasetautonomous vehiclescomputer visiondeep learninglidarmachine learningmappingrobotics An open multi-sensor dataset for autonomous driving research. This dataset comprises semantically segmented images, semantic point clouds, and 3D bounding boxes. In addition, it contains unlabelled 360 degree camera images, lidar, and bus data for three sequences. We hope this dataset will further facilitate active research and development in AI, computer vision, and robotics for autonomous driving. Details → Usage examples
See 1 usage example → ARPA-E PERFORM Forecast dataenergyenvironmentalgeospatialmodelsolarsustainability The ARPA-E PERFORM Program is an ARPA-E funded program that aim to use time-coincident power and load seeks to develop innovative management systems that represent the relative delivery risk of each asset and balance the collective risk of all assets across the grid. A risk-driven paradigm allows operators to: (i) fully understand the true likelihood of maintaining a supply-demand balance and system reliability, (ii) optimally manage the system, and (iii) assess the true value of essential reliability services. This paradigm shift is critical for all power systems and is essential for grids wi... Details → Usage examples
See 1 usage example → Allen Brain Observatory - Visual Coding AWS Public Data Setelectrophysiologyimage processingimaginglife sciencesMus musculusneurobiologyneuroimagingsignal processing The Allen Brain Observatory – Visual Coding is a large-scale, standardized survey of physiological activity across the mouse visual cortex, hippocampus, and thalamus. It includes datasets collected with both two-photon imaging and Neuropixels probes, two complementary techniques for measuring the activity of neurons in vivo. The two-photon imaging dataset features visually evoked calcium responses from GCaMP6-expressing neurons in a range of cortical layers, visual areas, and Cre lines. The Neuropixels dataset features spiking activity from distributed cortical and subcortical brain regions, c... Details → Usage examples
See 1 usage example → COVID-19 Genome Sequence DatasetbambioinformaticsbiologycoronavirusCOVID-19cramfastqgeneticgenomichealthlife sciencesMERSSARSSTRIDEStranscriptomicsviruswhole genome sequencing A centralized sequence repository for all records containing sequence associated with the novel corona virus (SARS-CoV-2) submitted to the National Center for Biotechnology Information (NCBI) Sequence Read Archive (SRA). Included are both the original sequences submitted by the principal investigator as well as SRA-processed sequences that require the SRA Toolkit for analysis. Additionally, submitter provided metadata included in associated BioSample and BioProject records is available alongside NCBI calculated data, such k-mer based taxonomy analysis results, contiguous assemblies (contigs) a... Details → Usage examples
See 1 usage example → Cell Painting Image Collectionbiologycell imagingcell paintingfluorescence imaginghigh-throughput imagingimaginglife sciencesmicroscopy The Cell Painting Image Collection is a collection of freely downloadable microscopy image sets. Cell Painting is an unbiased high throughput imaging assay used to analyze perturbations in cell models. In addition to the images themselves, each set includes a description of the biological application and some type of "ground truth" (expected results). Researchers are encouraged to use these image sets as reference points when developing, testing, and publishing new image analysis algorithms for the life sciences. We hope that the this data set will lead to a better understanding of w... Details → Usage examples
See 1 usage example → Coupled Model Intercomparison Project Phase 5 (CMIP5) University of Wisconsin-Madison Probabilistic Downscaling Datasetclimatecoastaldisaster responseenvironmentalmeteorologicaloceanssustainabilitywaterweather The University of Wisconsin Probabilistic Downscaling (UWPD) is a statistically downscaled dataset based on the Coupled Model Intercomparison Project Phase 5 (CMIP5) climate models. UWPD consists of three variables, daily precipitation and maximum and minimum temperature. The spatial resolution is 0.1°x0.1° degree resolution for the United States and southern Canada east of the Rocky Mountains. The downscaling methodology is not deterministic. Instead, to properly capture unexplained variability and extreme events, the methodology predicts a spatially and temporally varying Probability Density Function (PDF) for each variable. Statistics such as the mean, mean PDF and annual maximum statistics can be calculated directly from the daily PDF and these statistics are included in the dataset. In addition, “standard”, “raw” data is created by randomly sampling from the PDFs to create a “realization” of the local scale given the large-scale from the climate model. There are 3 realizations for temperature and 14 realizations for precipitation. ... Details → Usage examples
See 1 usage example → CoversBRcopyright monitoringcover song identificationlive song identificationmusicmusic features datasetmusic information retrievalmusic recognition CoversBR is the first large audio database with, predominantly, Brazilian music for the tasks of Covers Song Identification (CSI) and Live Song Identifications (LSI). Due to copyright restrictions audios of the songs cannot be made available, however metadata and files of features have public access. Audio streamings captured from radio and TV channels for the live song identification task will be made public. CoversBR is composed of metadata and features extracted from 102298 songs, distributed in 26366 groups of covers/versions, with an average of 3.88 versions per group. The entire collecti... Details → Usage examples
See 1 usage example → Daylight Map Distribution of OpenStreetMapdisaster responsegeospatialmappingosmsustainability Daylight is a complete distribution of global, open map data that’s freely available with support from community and professional mapmakers. Meta combines the work of global contributors to projects like OpenStreetMap with quality and consistency checks from Daylight mapping partners to create a free, stable, and easy-to-use street-scale global map. The Daylight Map Distribution contains a validated subset of the OpenStreetMap database. In addition to the standard OpenStreetMap PBF format, Daylight is available in two parquet formats that are optimized for AWS Athena including geometries (Poin... Details → Usage examples
See 1 usage example → Ford Multi-AV Seasonal Datasetautonomous vehiclescomputer visionlidarmappingroboticstransportationurbanweather This research presents a challenging multi-agent seasonal dataset collected by a fleet of Ford autonomous vehicles at different days and times during 2017-18. The vehicles The vehicles were manually driven on an average route of 66 km in Michigan that included a mix of driving scenarios like the Detroit Airport, freeways, city-centres, university campus and suburban neighbourhood, etc. Each vehicle used in this data collection is a Ford Fusion outfitted with an Applanix POS-LV inertial measurement unit (IMU), four HDL-32E Velodyne 3D-lidar scanners, 6 Point Grey 1.3 MP Cameras arranged on the... Details → Usage examples
See 1 usage example → Global Biodiversity Information Facility (GBIF) Species Occurrencesbiodiversitybioinformaticsconservationearth observationlife sciences The Global Biodiversity Information Facility (GBIF) is an international network and data infrastructure funded by the world's governments providing global data that document the occurrence of species. GBIF currently integrates datasets documenting over 1.6 billion species occurrences, growing daily. The GBIF occurrence dataset combines data from a wide array of sources including specimen-related data from natural history museums, observations from citizen science networks and environment recording schemes. While these data are constantly changing at GBIF.org, periodic snapshots are taken a... Details → Usage examples
See 1 usage example → High-Order Accurate Direct Numerical Simulation of Flow over a MTU-T161 Low Pressure Turbine Bladecomputational fluid dynamicsgreen aviationlow-pressure turbineturbulence The archive comprises snapshot, point-probe, and time-average data produced via a high-fidelity computational simulation of turbulent air flow over a low pressure turbine blade, which is an important component in a jet engine. The simulation was undertaken using the open source PyFR flow solver on over 5000 Nvidia K20X GPUs of the Titan supercomputer at Oak Ridge National Laboratory under an INCITE award from the US DOE. The data can be used to develop an enhanced understanding of the complex three-dimensional unsteady air flow patterns over turbine blades in jet engines. This could in turn le... Details → Usage examples
See 1 usage example → Human Cancer Models Initiative (HCMI) Cancer Model Development Centercancergenomiclife sciencesSTRIDESwhole genome sequencing The Human Cancer Models Initiative (HCMI) is an international consortium that is generating novel, next-generation, tumor-derived culture models annotated with genomic and clinical data. HCMI-developed models and related data are available as a community resource. The NCI is contributing to the initiative by supporting four Cancer Model Development Centers (CMDCs). CMDCs are tasked with producing next-generation cancer models from clinical samples. The cancer models include tumor types that are rare, originate from patients from underrepresented populations, lack precision therapy, or lack ca... Details → Usage examples
See 1 usage example → Legal Entity Identifier (LEI) and Legal Entity Reference Data (LE-RD)analyticsblockchainclimatecommercecopyright monitoringcsvfinancial marketsgovernancegovernment spendingjsonmarket datasocioeconomicstatisticstransparencyxml The Legal Entity Identifier (LEI) is a 20-character, alpha-numeric code based on the ISO 17442 standard developed by the International Organization for Standardization (ISO). It connects to key reference information that enables clear and unique identification of legal entities participating in financial transactions. Each LEI contains information about an entity’s ownership structure and thus answers the questions of 'who is who’ and ‘who owns whom’. Simply put, the publicly available LEI data pool can be regarded as a global directory, which greatly enhances transparency in the global ma... Details → Usage examples
See 1 usage example → NASA / USGS Europa Controlled Observationscogplanetarysatellite imagerystac The Solid State Imager (SSI) on NASA's Galileo spacecraft acquired more than 500 images of Jupiter's moon, Europa. These images vary from relatively low-resolution hemispherical imaging, to high-resolution targeted images that cover a small portion of the surface. Here we provide a set of 481 minimally processed, projected Galileo images with photogrammetrically improved locations on Europa's surface. These individual images were subsequently used as input into a set of 92 observation mosaics. These images provide users with nearly the entire Galileo Europa imaging dataset at its native resolution and with improved relative image locations. The Solid State Imager on NASA's Galileo spacecraft provided the only moderate- to high-resolution images of Jupiter's moon, Europa. Unfortunately, uncertainty in the position and pointing of the spacecraft, as well as the position and orientation of Europa, when the images were acquired resulted in significant errors in image locations on the surface. The result of these errors is that images acquired during different Galileo orbits, or even at different times during the same orbit, are significantly misaligned (errors of up to 100 km on the surface). The dataset provides a set of individual images that can be used for scientific analysis... Details → Usage examples
See 4 usage examples → NOAA Global Forecast System (GFS)agricultureclimatedisaster responseenvironmentalmeteorologicalsustainabilityweather The Global Forecast System (GFS) is a weather forecast model produced by the National Centers for Environmental Prediction (NCEP). Dozens of atmospheric and land-soil variables are available through this dataset, from temperatures, winds, and precipitation to soil moisture and atmospheric ozone concentration. The entire globe is covered by the GFS at a base horizontal resolution of 18 miles (28 kilometers) between grid points, which is used by the operational forecasters who predict weather out to 16 days in the future. Horizontal resolution drops to 44 miles (70 kilometers) between grid point for forecasts between one week and two weeks. The NOAA Global Forecast Systems (GFS) Warm Start Initial Conditions are produced by the National Centers for Environmental Prediction Center (NCEP) to run operational deterministic medium-range numerical weather
predictions. Details → Usage examples
See 1 usage example → NOAA Global Surface Summary of Dayagricultureclimateenvironmentalnatural resourceregulatorysustainabilityweather Global Surface Summary of the Day is derived from The Integrated Surface Hourly (ISH) dataset. The ISH dataset includes global data obtained from the USAF Climatology Center, located in the Federal Climate Complex with NCDC. The latest daily summary data are normally available 1-2 days after the date-time of the observations used in the daily summaries. The online data files begin with 1929 and are
at the time of this writing at the Version 8 software level. Over 9000 stations' data are typically available. The daily elements included in the dataset (as available from each station) are: G... Details → Usage examples
See 1 usage example → NOAA Integrated Surface Database (ISD)agricultureclimatemeteorologicalsustainabilityweather The Integrated Surface Database (ISD) consists of global hourly and synoptic observations compiled from numerous sources into a gzipped fixed width format. ISD was developed as a joint activity within Asheville's Federal Climate Complex. The database includes over 35,000 stations worldwide, with some having data as far back as 1901, though the data show a substantial increase in volume in the 1940s and again in the early 1970s. Currently, there are over 14,000 "active" stations updated daily in the database. The total uncompressed data volume is around 600 gigabytes; however, it ... Details → Usage examples
See 1 usage example → NOAA National Digital Forecast Database (NDFD)agricultureclimatemeteorologicalsustainabilityweather The National Digital Forecast Database (NDFD) is a suite of gridded forecasts of sensible weather elements (e.g., cloud cover, maximum temperature). Forecasts prepared by NWS field offices working in collaboration with the National Centers for Environmental Prediction (NCEP) are combined in the NDFD to create a seamless mosaic of digital forecasts from which operational NWS products are generated. The most recent data is under the opnl and expr prefixes. A copy is also placed under the wmo prefix. The wmo prefix is structured like so: wmo/<parameter>/<year>/<month>/<day&g... Details → Usage examples
See 1 usage example → NOAA/PMEL Ocean Climate Stations Mooringsclimateenvironmentaloceanssustainabilityweather The mission of the Ocean Climate Stations (OCS) Project is to make meteorological and oceanic measurements from autonomous platforms. Calibrated, quality-controlled, and well-documented climatological measurements are available on the OCS webpage and the OceanSITES Global Data Assembly Centers (GDACs), with near-realtime data available prior to release of the complete, downloaded datasets. OCS measurements served through the Big Data Program come from OCS high-latitude moored buoys located in the Kuroshio Extension (32°N 145°E) and the Gulf of Alaska (50°N 145°W). Initiated in 2004 and 20... Details → Usage examples
See 1 usage example → New Jersey Statewide Digital Aerial Imagery Catalogaerial imagerycogearth observationgeospatialimagingmapping The New Jersey Office of GIS, NJ Office of Information Technology manages a series of 11 digital orthophotography and scanned aerial photo maps collected at various years ranging from 1930 to 2017. Each year’s worth of imagery are available as Cloud Optimized GeoTIFF (COG) files and some years are available as compressed MrSID and/or JP2 files. Additionally, each year of imagery is organized into a tile grid scheme covering the entire geography of New Jersey. Many years share the same tiling grid while others have unique grids as defined by the project at the time. Details → Usage examples
See 1 usage example → New Jersey Statewide LiDARelevationgeospatiallidarmapping Elevation datasets in New Jersey have been collected over several years as several discrete projects. Each project covers a geographic area, which is a subsection of the entire state, and has differing specifications based on the available technology at the time and project budget. The geographic extent of one project may overlap that of a neighboring project. Each of the 18 projects contains deliverable products such as LAS (Lidar point cloud) files, unclassified/classified, tiled to cover project area; relevant metadata records or documents, most adhering to the Federal Geographic Data Com... Details → Usage examples
See 1 usage example → Ohio State Cardiac MRI Raw Data (OCMR)Homo sapiensimage processingimaginglife sciencesmagnetic resonance imagingsignal processing OCMR is an open-access repository that provides multi-coil k-space data for cardiac cine. The fully sampled MRI datasets are intended for quantitative comparison and evaluation of image reconstruction methods. The free-breathing, prospectively undersampled datasets are intended to evaluate their performance and generalizability qualitatively. Details → Usage examples
See 1 usage example → Oxford Nanopore Technologies Benchmark Datasetsbioinformaticsbiologyfast5fastqgenomicHomo sapienslife scienceswhole genome sequencing The ont-open-data registry provides reference sequencing data from Oxford Nanopore Technologies to support, 1) Exploration of the characteristics of nanopore sequence data. 2) Assessment and reproduction of performance benchmarks 3) Development of tools and methods. The data deposited showcases DNA sequences from a representative subset of sequencing chemistries. The datasets correspond to publicly-available reference samples (e.g. GM24385 as reference human). Raw data are provided with metadata and scripts to describe sample and data provenance. Details → Usage examples
See 1 usage example → SILAM Air Qualityair qualityclimateearth observationmeteorologicalsustainabilityweather Air Quality is a global SILAM atmospheric composition and air quality forecast performed on a daily basis for > 100 species and covering the troposphere and the stratosphere. The output produces 3D concentration fields and aerosol optical thickness. The data are unique: 20km resolution for global AQ models is unseen worldwide. Details → Usage examples
See 1 usage example → Sentinel-1 SLC dataset for Germanydisaster responseearth observationenvironmentalgeospatialsatellite imagerysustainabilitysynthetic aperture radar The Sentinel1 Single Look Complex (SLC) unzipped dataset contains Synthetic Aperture Radar (SAR) data from the European Space Agency’s Sentinel-1 mission. Different from the zipped data provided by ESA, this dataset allows direct access to individual swaths required for a given study area, thus drastically minimizing the storage and downloading time requirements of a project. Since the data is stored on S3, users can utilize the boto3 library and s3 get_object method to read the entire content of the object into the memory for processing, without actually having to download it. The Sentinel-1 ... Details → Usage examples
See 1 usage example → Tabula Murisbiologyencyclopedicgenomichealthlife sciencesmedicine Tabula Muris is a compendium of single cell transcriptomic data from the model organism Mus musculus comprising more than 100,000 cells from 20 organs and tissues. These data represent a new resource for cell biology, reveal gene expression in poorly characterized cell populations, and allow for direct and controlled comparison of gene expression in cell types shared between tissues, such as T-lymphocytes and endothelial cells from different anatomical locations. Two distinct technical approaches were used for most organs: one approach, microfluidic droplet-based 3’-end counting, enabled the s... Details → Usage examples
See 1 usage example → Voices Obscured in Complex Environmental Settings (VOiCES)automatic speech recognitiondenoisingmachine learningspeaker identificationspeech processing VOiCES is a speech corpus recorded in acoustically challenging settings, using distant microphone recording. Speech was recorded in real rooms with various acoustic features (reverb, echo, HVAC systems, outside noise, etc.). Adversarial noise, either television, music, or babble, was concurrently played with clean speech. Data was recorded using multiple microphones strategically placed throughout the room. The corpus includes audio recordings, orthographic transcriptions, and speaker labels. Details → Usage examples
See 1 usage example → 2021 Amazon Last Mile Routing Research Challenge Datasetamazon.scienceanalyticsdeep learninggeospatiallast milelogisticsmachine learningoptimizationroutingtransportationurban The 2021 Amazon Last Mile Routing Research Challenge was an innovative research initiative led by Amazon.com and supported by the Massachusetts Institute of Technology’s Center for Transportation and Logistics. Over a period of 4 months, participants were challenged to develop innovative machine learning-based methods to enhance classic optimization-based approaches to solve the travelling salesperson problem, by learning from historical routes executed by Amazon delivery drivers. The primary goal of the Amazon Last Mile Routing Research Challenge was to foster innovative applied research in r... Details → Usage examples
See 3 usage examples → A Realistic Cyber Defense Dataset (CSE-CIC-IDS2018)cyber securityinternetintrusion detectionnetwork traffic This dataset is the result of a collaborative project between the Communications Security Establishment (CSE) and The Canadian Institute for Cybersecurity (CIC) that use the notion of profiles to generate cybersecurity dataset in a systematic manner. It incluides a detailed description of intrusions along with abstract distribution models for applications, protocols, or lower level network entities. The dataset includes seven different attack scenarios, namely Brute-force, Heartbleed, Botnet, DoS, DDoS, Web attacks, and infiltration of the network from inside. The attacking infrastructure incl... Details → Australasian Genomesbiodiversitybiologyconservationgeneticgenomiclife sciencestranscriptomicswildlife Australasian Genomes is the genomic data repository for the Threatened Species Initiative (TSI) and the ARC Centre for Innovations in Peptide and Protein Science (CIPPS). This repository contains reference genomes, transcriptomes, resequenced genomes and reduced representation sequencing data from Australasian species. Australasian Genomes is managed by the Australasian Wildlife Genomics Group (AWGG) at the Univeristy of Sydney on behalf of our collaborators within TSI and CIPPS. Details → CAFE60 reanalysisclimatesustainability The CSIRO Climate retrospective Analysis and Forecast Ensemble system: version 1 (CAFE60v1) provides a large ensemble retrospective analysis of the global climate system from 1960 to present with sufficiently many realizations and at spatio-temporal resolutions suitable to enable probabilistic climate studies. Using a variant of the ensemble Kalman filter, 96 climate state estimates are generated over the most recent six decades. These state estimates are constrained by monthly mean ocean, atmosphere and sea ice observations such that their trajectories track the observed state while enabling ... Details → CCAFS-Climate Dataagricultureclimatefood securitysustainability High resolution climate data to help assess the impacts of climate change primarily on agriculture. These open access datasets of climate projections will help researchers make climate change impact assessments. Details → COCO - Common Objects in Context - fast.ai datasetscomputer visiondeep learningmachine learning COCO is a large-scale object detection, segmentation, and captioning dataset. This is part of the fast.ai datasets collection hosted by AWS for convenience of fast.ai students. If you use this dataset in your research please cite arXiv:1405.0312 [cs.CV]. Details → COVID-19 Molecular Structure and Therapeutics HubbioinformaticsbiologycoronavirusCOVID-19life sciencesmolecular dockingpharmaceutical Aggregating critical information to accelerate drug discovery for the molecular modeling and simulation community. A community-driven data repository and curation service for molecular structures, models, therapeutics, and simulations related to computational research related to therapeutic opportunities for COVID-19 (caused by the SARS-CoV-2 coronavirus). Details → Cloud to Street - Microsoft Flood and Clouds Datasetcogcomputer visiondeep learningearth observationfloodsgeospatialmachine learningsatellite imagerysynthetic aperture radar This dataset consists of chips of Sentinel-1 and Sentinel-2 satellite data. Each Sentinel-1 chip contains a corresponding label for water and each Sentinel-2 chip contains a corresponding label for water and clouds. Data is stored in folders by a unique event identifier as the folder name. Within each event folder there are subfolders for Sentinel-1 (s1) and Sentinel-2 (s2) data. Each chip is contained in its own sub-folder with the folder name being the source image id, followed by a unique chip identifier consisting of a hyphenated set of 5 numbers. All bands of the satellite data, as well a... Details → District of Columbia - Classified Point Cloud LiDARcitiesdisaster responsegeospatialus-dc LiDAR point cloud data for Washington, DC is available for anyone to use on Amazon S3. This dataset, managed by the Office of the Chief Technology Officer (OCTO), through the direction of the District of Columbia GIS program, contains tiled point cloud data for the entire District along with associated metadata. Details → Downscaled Climate Data for Alaskaagricultureclimatecoastalearth observationenvironmentalsustainabilityweather This dataset contains historical and projected dynamically downscaled climate data for the State of Alaska and surrounding regions at 20km spatial resolution and hourly temporal resolution. Select variables are also summarized into daily resolutions. This data was produced using the Weather Research and Forecasting (WRF) model (Version 3.5). We downscaled both ERA-Interim historical reanalysis data (1979-2015) and both historical and projected runs from 2 GCM’s from the Coupled Model Inter-comparison Project 5 (CMIP5): GFDL-CM3 and NCAR-CCSM4 (historical run: 1970-2005 and RCP 8.5: 2006-2100). Details → Epoch of Reionization Datasetastronomy The data are from observations with the Murchison Widefield Array (MWA) which is a Square Kilometer Array (SKA) precursor in Western Australia. This particular dataset is from the Epoch of Reionization project which is a key science driver of the SKA. Nearly 2PB of such observations have been recorded to date, this is a small subset of that which has been exported from the MWA data archive in Perth and made available to the public on AWS. The data were taken to detect signatures of the first stars and galaxies forming and the effect of these early stars and galaxies on the evolution of the u... Details → Galaxy Evolution Explorer Satellite (GALEX)astronomy The Galaxy Evolution Explorer Satellite (GALEX) was a NASA mission led by the California Institute of Technology, whose primary goal was to investigates how star formation in galaxies evolved from the early Universe up to the present. GALEX used microchannel plate detectors to obtain direct images in the near-UV (NUV) and far-UV (FUV), and a grism to disperse light for low resolution spectroscopy. More information about GALEX is available at MAST Details → Genome Arkbiodiversitybioinformaticsbiologyconservationgeneticgenomiclife sciences The Genome Ark hosts genomic information for the Vertebrate Genomes Project (VGP) and other related projects. The VGP is an international collaboration that aims to generate complete and near error-free reference genomes for all extant vertebrate species. These genomes will be used to address fundamental questions in biology and disease, to identify species most genetically at risk for extinction, and to preserve genetic information of life. Details → Google Books Ngramsnatural language processing N-grams are fixed size tuples of items. In this case the items are words extracted from the Google Books corpus. The n specifies the number of elements in the tuple, so a 5-gram contains five words or characters. The n-grams in this dataset were produced by passing a sliding window of the text of books and outputting a record for each new token. Details → HIRLAM Weather Modelagricultureclimateearth observationmeteorologicalsustainabilityweather HIRLAM (High Resolution Limited Area Model) is an operational synoptic and mesoscale weather prediction model managed by the Finnish Meteorological Institute. Details → High Resolution Downscaled Climate Data for Southeast Alaskaagricultureclimatecoastalearth observationenvironmentalsustainabilityweather This dataset contains historical and projected dynamically downscaled climate data for the Southeast region of the State of Alaska at 1 and 4km spatial resolution and hourly temporal resolution. Select variables are also summarized into daily resolutions. This data was produced using the Weather Research and Forecasting (WRF) model (Version 4.0). We downscaled both Climate Forecast System Reanalysis (CFSR) historical reanalysis data (1980-2019) and both historical and projected runs from two GCM’s from the Coupled Model Inter-comparison Project 5 (CMIP5): GFDL-CM3 and NCAR-CCSM4 (historical ru... Details → Homeland Security and Infrastructure US Citiesdisaster responseelevationgeospatiallidar The U.S. Cities elevation data collection program supported the US Department of Homeland Security Homeland Security and Infrastructure Program (HSIP). As part of the HSIP Program, there were 133+ U.S. cities that had imagery and LiDAR collected to provide the Homeland Security, Homeland Defense, and Emergency Preparedness, Response and Recovery (EPR&R) community with common operational, geospatially enabled baseline data needed to analyze threat, support critical infrastructure protection and expedite readiness, response and recovery in the event of a man-made or natural disaster. As a pa... Details → ISERVearth observationenvironmentalgeospatialsatellite imagerysustainability ISS SERVIR Environmental Research and Visualization System (ISERV) was a fully-automated prototype camera aboard the International Space Station that was tasked to capture high-resolution Earth imagery of specific locations at 3-7 frames per second. In the course of its regular operations during 2013 and 2014, ISERV's camera acquired images that can be used primaliry in use is environmental and disaster management. Details → Image localization - fast.ai datasetscomputer visiondeep learningmachine learning Some of the most important datasets for image localization research, including Camvid and PASCAL VOC (2007 and 2012). This is part of the fast.ai datasets collection hosted by AWS for convenience of fast.ai students. See documentation link for citation and license details for each dataset. Details → InRad COVID-19 X-Ray and CT ScansbioinformaticscoronavirusCOVID-19healthlife sciencesmedicineSARS This dataset is a collection of anonymized thoracic radiographs (X-Rays) and computed tomography (CT) scans of patients with suspected COVID-19. Images are acommpanied by a positive or negative diagnosis for SARS-CoV2 infection via RT-PCR. These images were provided by Hospital das Clínicas da Universidade de São Paulo, Hospital Sirio-Libanes, and by Laboratory Fleury. Details → K2 Mission Dataastronomy The K2 mission observed 100 square degrees for 80 days each across 20 different pointings along the ecliptic, collecting high-precision photometry for a selection of targets within each field. The mission began when the original Kepler mission ended due to loss of the second reaction wheel in 2011. More information about the K2 mission is available at MAST. Details → KITTI Vision Benchmark Suiteautonomous vehiclescomputer visiondeep learningmachine learningrobotics Dataset and benchmarks for computer vision research in the context of autonomous driving. The dataset has been recorded in and around the city of Karlsruhe, Germany using the mobile platform AnnieWay (VW station wagon) which has been equipped with several RGB and monochrome cameras, a Velodyne HDL 64 laser scanner as well as an accurate RTK corrected GPS/IMU localization unit. The dataset has been created for computer vision and machine learning research on stereo, optical flow, visual odometry, semantic segmentation, semantic instance segmentation, road segmentation, single image depth predic... Details → Kepler Mission Dataastronomy The Kepler mission observed the brightness of more than 180,000 stars near the Cygnus constellation at a 30 minute cadence for 4 years in order to find transiting exoplanets, study variable stars, and find eclipsing binaries. More information about the Kepler mission is available at MAST. Details → NLP - fast.ai datasetsdeep learningmachine learningnatural language processing Some of the most important datasets for NLP, with a focus on classification, including IMDb, AG-News, Amazon Reviews (polarity and full), Yelp Reviews (polarity and full), Dbpedia, Sogou News (Pinyin), Yahoo Answers, Wikitext 2 and Wikitext 103, and ACL-2010 French-English 10^9 corpus. This is part of the fast.ai datasets collection hosted by AWS for convenience of fast.ai students. See documentation link for citation and license details for each dataset. Details → NOAA Atmospheric Climate Data Recordsagricultureclimatemeteorologicalsustainabilityweather NOAA's Climate Data Records (CDRs) are robust, sustainable, and scientifically sound climate records that provide trustworthy information on how, where, and to what extent the land, oceans, atmosphere and ice sheets are changing. These datasets are thoroughly vetted time series measurements with the longevity, consistency, and continuity to assess and measure climate variability and change. NOAA CDRs are vetted using standards established by the National Research Council (NRC). Climate Data Records are created by merging data from surface, atmosphere, and space-based systems across decades. NOAA’s Climate Data Records provides authoritative and traceable long-term climate records. NOAA developed CDRs by applying modern data analysis methods to historical global satellite data. This process can clarify the underlying climate trends within the data and allows researchers and other users to identify economic and scientific value in these records. NCEI maintains and extends CDRs by applying the same methods to present-day and future satellite measurements. Atmospheric Climate Data Records are measurements of several global variables to help characterize the atmosphere... Details → NOAA Coastal Lidar Dataclimatedisaster responseelevationgeospatiallidarsustainability Lidar (light detection and ranging) is a technology that can measure the 3-dimentional location of objects, including the solid earth surface. The data consists of a point cloud of the positions of solid objects that reflected a laser pulse, typically from an airborne platform. In addition to the position, each point may also be attributed by the type of object it reflected from, the intensity of the reflection, and other system dependent metadata. The NOAA Coastal Lidar Data is a collection of lidar projects from many different sources and agencies, geographically focused on the coastal areas... Details → NOAA Continuously Operating Reference Stations (CORS) Network (NCN)broadcast ephemerisContinuously Operating Reference Station (CORS)earth observationgeospatialGNSSGPSmappingNOAA CORS Network (NCN)post-processingRINEXsurvey The NOAA Continuously Operating Reference Stations (CORS) Network (NCN), managed by NOAA/National Geodetic Survey (NGS), provide Global Navigation Satellite System (GNSS) data, supporting three dimensional positioning, meteorology, space weather, and geophysical applications throughout the United States. The NCN is a multi-purpose, multi-agency cooperative endeavor, combining the efforts of hundreds of government, academic, and private organizations. The stations are independently owned and operated. Each agency shares their GNSS/GPS carrier phase and code range measurements and station metadata with NGS, which are analyzed and distributed free of charge. ... Details → NOAA Fundamental Climate Data Records (FCDR)agricultureclimatemeteorologicalsustainabilityweather NOAA's Climate Data Records (CDRs) are robust, sustainable, and scientifically sound climate records that provide trustworthy information on how, where, and to what extent the land, oceans, atmosphere and ice sheets are changing. These datasets are thoroughly vetted time series measurements with the longevity, consistency, and continuity to assess and measure climate variability and change. NOAA CDRs are vetted using standards established by the National Research Council (NRC). Climate Data Records are created by merging data from surface, atmosphere, and space-based systems across decades. NOAA’s Climate Data Records provides authoritative and traceable long-term climate records. NOAA developed CDRs by applying modern data analysis methods to historical global satellite data. This process can clarify the underlying climate trends within the data and allows researchers and other users to identify economic and scientific value in these records. NCEI maintains and extends CDRs by applying the same methods to present-day and future satellite measurements. Fundamental CDRs are composed of sensor data (e.g. calibrated radiances, brightness temperatures) that have been ... Details → NOAA Global Ensemble Forecast System (GEFS)agricultureclimatemeteorologicalsustainabilityweather The Global Ensemble Forecast System (GEFS), previously known as the GFS Global ENSemble (GENS), is a weather forecast model made up of 21 separate forecasts, or ensemble members. The National Centers for Environmental Prediction (NCEP) started the GEFS to address the nature of uncertainty in weather observations, which is used to initialize weather forecast models. The GEFS attempts to quantify the amount of uncertainty in a forecast by generating an ensemble of multiple forecasts, each minutely different, or perturbed, from the original observations. With global coverage, GEFS is produced fo... Details → NOAA Global Extratropical Surge and Tide Operational Forecast System (Global ESTOFS)climatecoastaldisaster responseenvironmentalglobalmeteorologicaloceanssustainabilitywaterweather NOTICE - The Coast Survey Development Laboratory (CSDL) in NOAA/National Ocean Service (NOS)/Office of Coast Survey is proposing to upgrade the Surge and Tide Operational Forecast System (STOFS, formerly ESTOFS) to Version 1.0.1 in late fall of 2022. CSDL is seeking comments on this proposed upgrade through September 1, 2022. If approved, a Service Change Notice (SCN) will be issued at least 30 days before implementation of STOFS V1.0.1 with more detailed information. More details on the Public Information Statement can be found "HERE" NOAA's Global Extratropical Surge and Tide Operational Forecast System (Global ESTOFS) provides users with nowcasts (analyses of near present conditions) and forecast guidance of water level conditions for the entire globe. Global ESTOFS has been developed to serve the marine navigation, weather forecasting, and disaster mitigation user communities. Global ESTOFS was developed in a collaborative effort between the NOAA/National Ocean Service (NOS)/Office of Coast Survey, the NOAA/National Weather Service (NWS)/National Centers for Environmental Prediction (NCEP) Central Operations (NCO), the University of Notre Dame, the University of North Carolina, and The Water Institute of the Gulf. The model generates forecasts out to 180 hours four times per day; forecast output includes water levels caused by the combined effects of storm surge and tides, by astronomical tides alone, and by sub-tidal water levels (isolated storm surge). The hydrodynamic model employed by Global ESTOFS is the ADvanced CIRCulation (ADCIRC) finite element model. The model is forced by GFS winds, mean sea level pressure, and sea ice. The unstructured grid used by Global ESTOFS consists of 8,452,486 nodes and 16,226,163 triangular elements. Coastal resolution is up to 80 m for Hawaii and the U.S. West Coast; up to 90-120 m for the Pacific Islands including Guam, American Samoa, Marianas, Wake Island, Marshall Islands, and Palau; and up to 120 m for the U.S. East Coast, Puerto Rico, Micronesia, and Alaska. The flood plain extends overland to approximately 6 m elevation ASL for the U.S. East Coast, and up to 20 m elevation ASL for the Pacific Islands. Global ESTOFS a) reduces bias and errors due to the removal of the open ocean boundaries that were included in previous ESTOFS regional domains (ESTOFS-Atlantic, -Pacific, -Micronesia); b) includes internal tide-induced dissipation in the deep ocean; c) includes sea ice effect on wind drag, and d) incorporates a bias correction using 2-day average water level observations from CO-OPS tide stations that are interpolated spatially across the Global ESTOFS mesh. Global ESTOFS water level forecast guidance outpu... Details → NOAA Global Hydro Estimator (GHE)agriculturemeteorologicalsustainabilitywaterweather Global Hydro-Estimator provides a global mosaic imagery of rainfall estimates from multi-geostationary satellites, which currently includes GOES-16, GOES-15, Meteosat-8, Meteosat-11 and Himawari-8. The GHE products include: Instantaneous rain rate, 1 hour, 3 hour, 6 hour, 24 hour and also multi-day rainfall accumulation. Details → NOAA Global Mosaic of Geostationary Satellite Imagery (GMGSI)agricultureclimatemeteorologicalsustainabilityweather NOAA/NESDIS Global Mosaic of Geostationary Satellite Imagery (GMGSI) visible (VIS), shortwave infrared (SIR), longwave infrared (LIR) imagery, and water vaport imagery (WV) are composited from data from several geostationary satellites orbiting the globe, including the GOES-East and GOES-West Satellites operated by U.S. NOAA/NESDIS, the Meteosat-11 and Meteosat-8 satellites from theMeteosat Second Generation (MSG) series of satellites operated by European Organization for the Exploitation of Meteorological Satellites (EUMETSAT), and the Himawari-8 satellite operated by the Japan Meteorological... Details → NOAA Global Real-Time Ocean Forecast System (Global RTOFS)climatecoastaldisaster responseenvironmentalglobalmeteorologicaloceanssustainabilitywaterweather NOAA's Global Real-Time Ocean Forecast System (Global RTOFS) provides users with nowcasts (analyses of near present conditions) and forecast guidance up to eight days of ocean temperature and salinity, water velocity, sea surface elevation, sea ice coverage and sea ice thickness. The Global Operational Real-Time Ocean Forecast System (Global RTOFS) is based on an eddy resolving 1/12° global HYCOM (HYbrid Coordinates Ocean Model) (https://www.hycom.org/), which is coupled to the Community Ice CodE (CICE) Version 4 (https://www.arcus.org/witness-the-arctic/2018/5/highlight/1). The RTOFS grid has a 1/12 degree horizontal resolution and 41 hybrid vertical levels on a global tripolar grid. Since 2020, the RTOFS system implements a multivariate, multi-scale 3DVar data assimilation algorithm (Cummings and Smedstad, 2014) using a 24-hour update cycle. The data types presently assimilated include (1) satellite Sea Surface Temperature (SST) from METOP-B, JPSS-VIIRS, and in-Situ SST, from ships, fixed and drifting buoys The system is designed to incorporate new observing systems as the data becomes available. Once the observations go through a fully automated quality control and thinning process, the increments, or corrections, are obtained by executing the 3D variational algorithm. The increments are then added to the 24-hours forecast fields using a 6-hourly incremental analysis update. An earlier version of the system is described in Garraffo et al (2020). Garraffo, Z.D., J.A. Cummings, S. Paturi, Y. Hao, D. Iredell, T. Spindler, B. Balasubramanian, I. Rivin, H-C. Kim, A. Mehra, 2020. Real Time Ocean-Sea Ice Coupled Three Dimensional Variational Global Data Assimilative Ocean Forecast System. In Research Activities in Earth System Modeling, edited by E. Astakhova, WMO, World Climate Research Program Report No.6, July 2020. Cummings, J. A. and O. M. Smedstad. 2013. Variational Data Assimilation for the Global Ocean. Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications (Vol II) S. Park and L. Xu (eds), Springer, Chapter 13, 303-343. Global Real ... Details → NOAA National Bathymetric Source Databathymetryearth observationmarine navigationmodeloceansoceans The National Bathymetric Source (NBS) project creates and maintains high-resolution bathymetry composed of the best available data. This project enables the creation of next-generation nautical charts while also providing support for modeling, industry, science, regulation, and public curiosity. Primary sources of bathymetry include NOAA and U.S. Army Corps of Engineers hydrographic surveys and topographic bathymetric (topo-bathy) lidar (light detection and ranging) data. Data submitted through the NOAA Office of Coast Survey’s external source data process are also included, with gaps... Details → NOAA National Blend of Models (NBM)agricultureclimatecogmeteorologicalsustainabilityweather The National Blend of Models (NBM) is a nationally consistent and skillful suite of calibrated forecast guidance based on a blend of both NWS and non-NWS numerical weather prediction model data and post-processed model guidance. The goal of the NBM is to create a highly accurate, skillful and consistent starting point for the gridded forecast. Details → NOAA National Water Model Short-Range Forecastagricultureagricultureclimatedisaster responseenvironmentalsustainabilitytransportationweather The National Water Model (NWM) is a water resources model that simulates and forecasts water budget variables, including snowpack, evapotranspiration, soil moisture and streamflow, over the entire continental United States (CONUS). The model, launched in August 2016, is designed to improve the ability of NOAA to meet the needs of its stakeholders (forecasters, emergency managers, reservoir operators, first responders, recreationists, farmers, barge operators, and ecosystem and floodplain managers) by providing expanded accuracy, detail, and frequency of water information. It is operated by NOA... Details → NOAA North American Mesoscale Forecast System (NAM)agricultureclimatemeteorologicalsustainabilityweather The North American Mesoscale Forecast System (NAM) is one of the National Centers For Environmental Prediction’s (NCEP) major models for producing weather forecasts. NAM generates multiple grids (or domains) of weather forecasts over the North American continent at various horizontal resolutions. Each grid contains data for dozens of weather parameters, including temperature, precipitation, lightning, and turbulent kinetic energy. NAM uses additional numerical weather models to generate high-resolution forecasts over fixed regions, and occasionally to follow significant weather events like hur... Details → NOAA Oceanic Climate Data Recordsagricultureclimatemeteorologicaloceanssustainabilityweather NOAA's Climate Data Records (CDRs) are robust, sustainable, and scientifically sound climate records that provide trustworthy information on how, where, and to what extent the land, oceans, atmosphere and ice sheets are changing. These datasets are thoroughly vetted time series measurements with the longevity, consistency, and continuity to assess and measure climate variability and change. NOAA CDRs are vetted using standards established by the National Research Council (NRC). Climate Data Records are created by merging data from surface, atmosphere, and space-based systems across decades. NOAA’s Climate Data Records provides authoritative and traceable long-term climate records. NOAA developed CDRs by applying modern data analysis methods to historical global satellite data. This process can clarify the underlying climate trends within the data and allows researchers and other users to identify economic and scientific value in these records. NCEI maintains and extends CDRs by applying the same methods to present-day and future satellite measurements. Oceanic Climate Data Records are measurements of oceans and seas both surface and subsurface as well as frozen st... Details → NOAA Rapid Refresh (RAP)agricultureclimatemeteorologicalsustainabilityweather The Rapid Refresh (RAP) is a NOAA/NCEP operational weather prediction system comprised primarily of a numerical forecast model and analysis/assimilation system to initialize that model. It covers North America and is run with a horizontal resolution of 13 km and 50 vertical layers. The RAP was developed to serve users needing frequently updated short-range weather forecasts, including those in the US aviation community and US severe weather forecasting community. The model is run for every hour of the day; it is integrated to 51 hours for the 03/09/15/21 UTC cycles and to 21 hours for every ot... Details → NOAA Real-Time Mesoscale Analysis (RTMA)agricultureclimatemeteorologicalsustainabilityweather The Real-Time Mesoscale Analysis (RTMA) is a NOAA National Centers For Environmental Prediction (NCEP) high-spatial and temporal resolution analysis/assimilation system for near-surf ace weather conditions. Its main component is the NCEP/EMC Gridpoint Statistical Interpolation (GSI) system applied in two-dimensional variational mode to assimilate conventional and satellite-derived observations. The RTMA was developed to support NDFD operations and provide field forecasters with high quality analyses for nowcasting, situational awareness, and forecast verification purposes. The system produces ... Details → NOAA Severe Weather Data Inventory (SWDI)agricultureclimatemeteorologicalsustainabilityweather The Storm Events Database is an integrated database of severe weather events across the United States from 1950 to this year, with information about a storm event's location, azimuth, distance, impact, and severity, including the cost of damages to property and crops. It contains data documenting: The occurrence of storms and other significant weather phenomena having sufficient intensity to cause loss of life, injuries, significant property damage, and/or disruption to commerce. Rare, unusual, weather phenomena that generate media attention, such as snow flurries in South Florida or the S... Details → NOAA Space Weather Forecast and Observation Dataclimatemeteorologicalsolarsustainabilityweather Space weather forecast and observation data is collected and disseminated by NOAA’s Space Weather Prediction Center (SWPC) in Boulder, CO. SWPC produces forecasts for multiple space weather phenomenon types and the resulting impacts to Earth and human activities. A variety of products are available that provide these forecast expectations, and their respective measurements, in formats that range from detailed technical forecast discussions to NOAA Scale values to simple bulletins that give information in laymen's terms. Forecasting is the prediction of future events, based on analysis and... Details → NOAA Terrestrial Climate Data Recordsagricultureclimatemeteorologicalsustainabilityweather NOAA's Climate Data Records (CDRs) are robust, sustainable, and scientifically sound climate records that provide trustworthy information on how, where, and to what extent the land, oceans, atmosphere and ice sheets are changing. These datasets are thoroughly vetted time series measurements with the longevity, consistency, and continuity to assess and measure climate variability and change. NOAA CDRs are vetted using standards established by the National Research Council (NRC). Climate Data Records are created by merging data from surface, atmosphere, and space-based systems across decades. NOAA’s Climate Data Records provides authoritative and traceable long-term climate records. NOAA developed CDRs by applying modern data analysis methods to historical global satellite data. This process can clarify the underlying climate trends within the data and allows researchers and other users to identify economic and scientific value in these records. NCEI maintains and extends CDRs by applying the same methods to present-day and future satellite measurements. Terrestrial CDRs are composed of sensor data that have been improved and quality controlled over time, together w... Details → NOAA U.S. Climate Gridded Dataset (NClimGrid)agricultureclimatemeteorologicalsustainabilityweather The NOAA Monthly U.S. Climate Gridded Dataset (NClimGrid) consists of four climate variables derived from the GHCN-D dataset: maximum temperature, minimum temperature, average temperature and precipitation. Each file provides monthly values in a 5x5 lat/lon grid for the Continental United States. Data is available from 1895 to the present. On an annual basis, approximately one year of "final" nClimGrid will be submitted to replace the initially supplied "preliminary" data for the same time period. Users should be sure to ascertain which level of data is required for their research. EpiNOAA is an analysis ready dataset that consists of a daily time-series of nClimGrid measures (maximum temperature, minimum temperature, average temperature, and precipitation) at the county scale. Each file provides daily values for the Continental United States. Data are available from 1951 to the present. Daily data are updated every 3 days with a preliminary data file and replaced with the scaled (i.e., quality controlled) data file every three months. This derivative data product is an enhancement from the original daily nClimGrid dataset in that all four weather parameters are now p... Details → NOAA Unified Forecast System (UFS) Marine Reanalysis: 1979-2019agricultureclimatemeteorologicalsustainabilityweather The NOAA UFS Marine Reanalysis is a global sea ice ocean coupled reanalysis product produced by the marine data assimilation team of the UFS Research-to-Operation (R2O) project. Underlying forecast and data assimilation systems are based on the UFS model prototype version-6 and the Next Generation Global Ocean Data Assimilation System (NG-GODAS) release of the Joint Effort for Data assimilation Integration (JEDI) Sea Ice Ocean Coupled Assimilation (SOCA). Covering the 40 year reanalysis time period from 1979 to 2019, the data atmosphere option of the UFS coupled global atmosphere ocean sea ice (DATM-MOM6-CICE6) model was applied with two atmospheric forcing data sets: CFSR from 1979 to 1999 and GEFS from 2000 to 2019. Assimilated observation data sets include extensive space-based marine observations and conventional direct measurements of in situ profile data sets. This first UFS-marine interim reanalysis product is released to the broader weather and earth system modeling and analysis communities to obtain scientific feedback and applications for the development of the next generation operational numerical weather prediction system at the National Weather Service(NWS). The released file sets include two parts 1.) 1979 - 2019 UFS-DATM-MOM6-CICE6 model free runs and 2) 1979-2019 reanalysis cycle outputs (see descriptions embedded in each file set). Analyzed sea ice and ocean variables are ocean temperature, salinity, sea surface height, and sea ice conce... Details → NOAA Unified Forecast System Short-Range Weather (UFS SRW) Applicationagricultureclimatemeteorologicalsustainabilityweather The "Unified Forecast System (UFS)" is a community-based, coupled, comprehensive Earth Modeling System. It supports " multiple applications" with different forecast durations and spatial domains. The UFS Short-Range Weather (SRW) Application figures among these applications. It targets predictions of atmospheric behavior on a limited spatial domain and on time scales from minutes to several days. The SRW Application includes a prognostic atmospheric model, pre-processor, post-processor, and community workflow for running the system end-to-end. The "SRW Application Users's Guide" includes information on these components and provides detailed instructions on how to build and run the SRW Application. Users can access additional technical support via the "UFS Community Forum" This data registry contains the data required to run the “out-of-the-box” SRW Application case. The SRW App requires numerous input files to run, including static datasets (fix files containing climatological information, terrain and land use data), initial condition data files, lateral boundary condition data files, and model configuration files (such as namelists). The SRW App experiment generation system also contains a set of workflow end-to-end (WE2E) tests that exercise various configurations of the system (e.g., different grids, physics suites). Data for running a subset of these WE2E tests are also included within this registry. Users can generate forecasts for dates not included in this data registry by downloading and manually adding raw model files for the desired dates. Many of these model files are publicly available and can be accessed via links on the "Developmental Testbed Center&... Details → NOAA Unified Forecast System Subseasonal to Seasonal Prototypesagricultureclimatedisaster responseenvironmentalmeteorologicaloceanssustainabilityweather The Unified Forecast System Subseasonal to Seasonal prototypes consist of reforecast data from the UFS atmosphere-ocean coupled model experimental prototype version 5, 6, 7, and 8 produced by the Medium Range and Subseasonal to Seasonal Application team of the UFS-R2O project. The UFS prototypes are the first dataset released to the broader weather community for analysis and feedback as part of the development of the next generation operational numerical weather prediction system from NWS. The datasets includes all the major weather variables for atmosphere, land, ocean, sea ice, and ocean wav... Details → NOAA Unified Forecast System Weather Model (UFS-WM) Regression Testsagricultureclimatemeteorologicalsustainabilityweather The Unified Forecast System (UFS) is a community-based, coupled, comprehensive Earth Modeling System. The ufs-weather-model (UFS-WM) is the model source of the UFS for NOAA’s operational numerical weather prediction applications. The UFS-WM Regression Test (RT) is the testing software to ensure that previously developed and tested capabilities in UFS-WM still work after code changes are integrated into the system. It is required that UFS-WM RTs are performed successfully on the required Tier-1 platforms whenever code changes are made to the UFS-WM. The results of the UFS-WM RTs are summarized in log files and these files will be committed to the UFS-WM repository along with the code changes. Currently, the UFS-WM RTs have been developed to support several applications targeted for operational implementations including the global weather forecast, subseasonal to seasonal forecasts, hurricane forecast, regional rapid refresh forecast, and ocean analysis. At this time, there are 123 regression tests to support the UFS applications. The tests are evolving along with the development merged to the UFS-WM code repository. The regression test framework has been developed in the UFS-WM to run these tests on tier-1 supported systems. Each of the regression tests require a set of input data files and configuration files. The configuration files include namelist and model configuration files residing within the UFS-WM code repository. The input data includes initial conditions, climatology data, and fixed data sets such as orographic data and grid sp... Details → Nanopore Reference Human Genomegeneticgenomiclife scienceswhole genome sequencing This dataset includes the sequencing and assembly of a reference standard human genome (GM12878) using the MinION nanopore sequencing instrument with the R9.4 1D chemistry. Details → Natural Scenes Datasetcomputer visionimage processingimaginglife sciencesmachine learningmagnetic resonance imagingneuroimagingneurosciencenifti Here, we collected and pre-processed a massive, high-quality 7T fMRI dataset that can be used to advance our understanding of how the brain works. A unique feature of this dataset is the massive amount of data available per individual subject. The data were acquired using ultra-high-field fMRI (7T, whole-brain, 1.8-mm resolution, 1.6-s TR). We measured fMRI responses while each of 8 participants viewed 9,000–10,000 distinct, color natural scenes (22,500–30,000 trials) in 30–40 weekly scan sessions over the course of a year. Additional measures were collected including resting-state data, retin... Details → OpenFold Training Dataalphafoldlife sciencesmsaopen source softwareopenfoldproteinprotein foldingprotein template Multiple sequence alignments (MSAs) for 132,000 unique Protein Data Bank (PDB) chains, covering 640,000 PDB chains in total, and 4,850,000 UniClust30 clusters. Template hits are also provided for the PDB chains and 270,000 UniClust30 clusters chosen for maximal diversity and MSA depth. MSAs were generated with HHBlits (-n3) and JackHMMER against MGnify, BFD, UniRef90, and UniClust30 while templates were identified from PDB70 with HHSearch, all according to procedures outlined in the supplement to the AlphaFold 2 Nature paper, Jumper et al. 2021. We expect the database to be broadly useful to s... Details → OpenNeurobiologyimaginglife sciencesneurobiologyneuroimaging OpenNeuro is a database of openly-available brain imaging data. The data are shared according to a Creative Commons CC0 license, providing a broad range of brain imaging data to researchers and citizen scientists alike. The database primarily focuses on functional magnetic resonance imaging (fMRI) data, but also includes other imaging modalities including structural and diffusion MRI, electroencephalography (EEG), and magnetoencephalograpy (MEG). OpenfMRI is a project of the Center for Reproducible Neuroscience at Stanford University. Development of the OpenNeuro resource has been funded by th... Details → PROJ datum gridsgeospatialmapping Horizontal and vertical adjustment datasets for coordinate transformation to be used by PROJ 7 or later. PROJ is a generic coordinate transformation software that transforms geospatial coordinates from one coordinate reference system (CRS) to another. This includes cartographic projections as well as geodetic transformations. Details → Physionetbiologylife sciences PhysioNet offers free web access to large collections of recorded physiologic signals (PhysioBank) and related open-source software (PhysioToolkit). Details → Provision of Web-Scale Parallel Corpora for Official European Languages (ParaCrawl)machine translationnatural language processing ParaCrawl is a set of large parallel corpora to/from English for all official EU languages by a broad web crawling effort. State-of-the-art methods are applied for the entire processing chain from identifying web sites with translated text all the way to collecting, cleaning and delivering parallel corpora that are ready as training data for CEF.AT and translation memories for DG Translation. Details → SMN Hi-Res Weather Forecast over Argentinaearth observationmeteorologicalnatural resourcesustainabilityweather The Servicio Meteorológico Nacional de Argentina (SMN-Arg), the National Meteorological Service of Argentina, shares its deterministic forecasts generated with WRF 4.0 (Weather and Research Forecasting) initialized at 00 and 12 UTC every day. This forecast includes some key hourly surface variables –2 m temperature, 2 m relative humidity, 10 m wind magnitude and direction, and precipitation–, along with other daily variables, minimum and maximum temperature. The forecast covers Argentina, Chile, Uruguay, Paraguay and parts of Bolivia and Brazil in a Lambert conformal projection, with 4 km... Details → SUCHO Ukrainian Cultural Heritage Web Archivescultural preservationinternetukraine The dataset contains web archives of Open Access collections of digitised cultural heritage from more than 3,000+ websites of Ukrainian cultural institutions, such as museums, libraries or archives. The web archives have been produced by SUCHO, which is a volunteer group of more than 1,300 international cultural heritage professionals – librarians, archivists, researchers, programmers - who have joined forces to save as much digitised cultural heritage during the 2022 invasion of Ukraine before the servers hosting them get destroyed, damaged or go offline for any other reason. The web archives... Details → Smithsonian Open Accessartcultureencyclopedichistorymuseum The Smithsonian’s mission is the "increase and diffusion of knowledge" and has been collecting since 1846. The Smithsonian, through its efforts to digitize its multidisciplinary collections, has created millions of digital assets and related metadata describing the collection objects. On February 25th, 2020, the Smithsonian released over 2.8 million CC0 interdisciplinary 2-D and 3-D images, related metadata, and additionally, research data from researches across the Smithsonian. The 2.8 million "open access" collections are a subset of the Smithsonian’s 155 million objects,... Details → Software Heritage Graph Datasetdigital preservationfree softwareopen source softwaresource code Software Heritage is the largest existing public archive of software source code and accompanying development history. The Software Heritage Graph Dataset is a fully deduplicated Merkle DAG representation of the Software Heritage archive.The dataset links together file content identifiers, source code directories, Version Control System (VCS) commits tracking evolution over time, up to the full states of VCS repositories as observed by Software Heritage during periodic crawls. The dataset’s contents come from major development forges (including GitHub and GitLab), FOSS distributions (e.g., Deb... Details → Tabula Muris Senisbiologyencyclopedicgenomichealthlife sciencesmedicinesingle-cell transcriptomics Tabula Muris Senis is a comprehensive compendium of single cell transcriptomic data from the model organism Mus musculus comprising more than 500,000 cells from 18 organs and tissues across the mouse lifespan. We discovered cell-specific changes occurring across multiple cell types and organs, as well as age related changes in the cellular composition of different organs. Using single-cell transcriptomic data we were able to assess cell type specific manifestations of different hallmarks of aging, such as senescence, changes in the activity of metabolic pathways, depletion of stem-cell populat... Details → Tabula Sapiensbiologyencyclopedicgeneticgenomichealthlife sciencesmedicinesingle-cell transcriptomics Tabula Sapiens will be a benchmark, first-draft human cell atlas of two million cells from 25 organs of eight normal human subjects. Taking the organs from the same individual controls for genetic background, age, environment, and epigenetic effects, and allows detailed analysis and comparison of cell types that are shared between tissues. Our work creates a detailed portrait of cell types as well as their distribution and variation in gene expression across tissues and within the endothelial, epithelial, stromal and immune compartments. A critical factor in the Tabula projects is our large collaborative network of PI’s with deep expertise at preparation of diverse organs, enabling all organs from a subject to be successfully processed within a single day. Tabula Sapiens leverages our network of human tissue experts and a close collaboration with a Donor Network West, a not-for-profit organ procurement organization. We use their experience to balance and assign cell types from each tissue compartment and optimally mix high-quality plate-seq data and high-volume droplet-based data to provide a broad and deep benchmark atlas. Our goal is to make sequence data rapidly and broadly available to the scientific community as a community resource. Before you use our data, please take note of our Data Release Policy below. Data Release Policy Our goal is to make sequence data rapidly and broadly available to the scientific community as a community resource. It is our intention to publish the work of this project in a timely fashion, and we welcome collaborative interaction on the project and analyses. However, considerable investment was made in generating these data and we ask that you respect rights of first publication and acknowledgment as outlined in the Toronto agreement. By accessing these data, you agree not to publish any articles containing analyses of genes, cell types or transcriptomic data on a who... Details → The Genome Modeling Systemgeneticgenomiclife sciences The Genome Institute at Washington University has developed a high-throughput, fault-tolerant analysis information management system called the Genome Modeling System (GMS), capable of executing complex, interdependent, and automated genome analysis pipelines at a massive scale. The GMS framework provides detailed tracking of samples and data coupled with reliable and repeatable analysis pipelines. GMS includes a full system image with software and services, expandable from one workstation to a large compute cluster. Details → The Massively Multilingual Image Dataset (MMID)computer visionmachine learningmachine translationnatural language processing MMID is a large-scale, massively multilingual dataset of images paired with the words they represent collected at the University of Pennsylvania. The dataset is doubly parallel: for each language, words are stored parallel to images that represent the word, and parallel to the word's translation into English (and corresponding images.) Details → UCSC Genome Browser Sequence and Annotationsbioinformaticsbiologygeneticgenomiclife sciences The UCSC Genome Browser is an online graphical viewer for genomes, a genome browser, hosted by the University of California, Santa Cruz (UCSC). The interactive website offers access to genome sequence data from a variety of vertebrate and invertebrate species and major model organisms, integrated with a large collection of aligned annotations. This dataset is a copy of the MySQL tables in MyISAM binary and tab-sep format and all binary files in custom formats, sometimes referred as 'gbdb'-files. Data from the UCSC Genome Browser is free and open for use by anyone. However, every genome... Details → University of British Columbia Sunflower Genome Datasetagriculturebiodiversitybioinformaticsbiologyfood securitygeneticgenomiclife scienceswhole genome sequencing This dataset captures Sunflower's genetic diversity originating from thousands of wild, cultivated, and landrace sunflower individuals distributed across North America.The data consists of raw sequences and associated botanical metadata, aligned sequences (to three different reference genomes), and sets of SNPs computed across several cohorts. Details → iNaturalist Licensed Observation Imagesbiodiversitybioinformaticsconservationearth observationlife sciences iNaturalist is a community science effort in which participants share observations of living organisms that they encounter and document with photographic evidence, location, and date. The community works together reviewing these images to identify these observations to species. This collection represents the licensed images accompanying iNaturalist observations. Details → stdpopsim species resourcesgenetic mapslife sciencespopulation geneticsrecombination mapssimulations Contains all resources (genome specifications, recombination maps, etc.) required for species specific simulation with the stdpopsim package. These resources are originally from a variety of other consortium and published work but are consolidated here for ease of access and use. If you are interested in adding a new species to the stdpopsim resource please raise an issue on the stdpopsim GitHub page to have the necessary files added here. Details → AgricultureVisionaerial imageryagriculturecomputer visiondeep learningmachine learning Agriculture-Vision aims to be a publicly available large-scale aerial agricultural image dataset that is high-resolution, multi-band, and with multiple types of patterns annotated by agronomy experts. The original dataset affiliated with the 2020 CVPR paper includes 94,986 512x512images sampled from 3,432 farmlands with nine types of annotations: double plant, drydown, endrow, nutrient deficiency, planter skip, storm damage, water, waterway and weed cluster. All of these patterns have substantial impacts on field conditions and the final yield. These farmland images were captured between 201... Details → Usage examples
See 2 usage examples → ChEMBL - Data Lakehouse Readybiotech blueprintchemistrygenomiclife sciencesmoleculeparquet ChEMBL is a manually curated database of bioactive molecules with drug-like properties. It brings together chemical, bioactivity and genomic data to aid the translation of genomic information into effective new drugs. This representation of ChEMBL is stored in Parquet format and most easily utilized through Amazon Athena. Follow the documentation for install instructions (< 2 minute install). New ChEMBL releases occur sporadically; the most up to date information on ChEMBL releases can be found here. Details → Usage examples
See 2 usage examples → ClinVar - Data Lakehouse Readybiotech blueprintchemistrygeneticgenomiclife sciencesparquet ClinVar is a freely accessible, public archive of reports of the relationships among human variations and phenotypes, with supporting evidence. ClinVar thus facilitates access to and communication about the relationships asserted between human variation and observed health status, and the history of that interpretation. ClinVar processes submissions reporting variants found in patient samples, assertions made regarding their clinical significance, information about the submitter, and other supporting data. The alleles described in submissions are mapped to reference sequences, and reported acc... Details → Usage examples
See 2 usage examples → NASA Earth Exchange Global Daily Downscaled Projections (NEX-GDDP-CMIP6)air temperatureclimateclimate modelclimate projectionsCMIP6cogearth observationenvironmentalglobalmodelNASA Center for Climate Simulation (NCCS)near-surface relative humiditynear-surface specific humiditynetcdfprecipitation The NEX-GDDP-CMIP6 dataset is comprised of global downscaled climate scenarios derived from the General Circulation Model (GCM) runs conducted under the Coupled Model Intercomparison Project Phase 6 (CMIP6) and across two of the four "Tier 1" greenhouse gas emissions scenarios known as Shared Socioeconomic Pathways (SSPs). The CMIP6 GCM runs were developed in support of the Sixth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC AR6). This dataset includes downscaled projections from ScenarioMIP model runs for which daily scenarios were produced and distributed... Details → Usage examples
See 2 usage examples → YouTube 8 Million - Data Lakehouse Readycomputer visionlabeledmachine learningparquetvideo This both the original .tfrecords and a Parquet representation of the YouTube 8 Million dataset. YouTube-8M is a large-scale labeled video dataset that consists of millions of YouTube video IDs, with high-quality machine-generated annotations from a diverse vocabulary of 3,800+ visual entities. It comes with precomputed audio-visual features from billions of frames and audio segments, designed to fit on a single hard disk. This dataset also includes the YouTube-8M Segments data from June 2019. This dataset is 'Lakehouse Ready'. Meaning, you can query this data in-place straight out of... Details → Usage examples
See 2 usage examples → 1000 Genomes Phase 3 Reanalysis with DRAGEN 3.5 - Data Lakehouse ReadybioinformaticsbiologygeneticgenomicHomo sapienslife sciencesparquetpopulation geneticsvcf The 1000 Genomes Project is an international collaboration which has established the most detailed catalogue of human genetic variation, including SNPs, structural variants, and their haplotype context. There were a total of 3202 individuals sequenced as part of Phase 3 of this project. The high coverage samples were processed using the Illumina DRAGEN v3.5.7b pipeline and are available at s3://1000genomes-dragen/. This dataset contains the VCFs transformed to Parquet/ORC in 3 different schemas - partitioned by samples, partitioned by chromosome and a nested data format. These representations ... Details → Usage examples
See 1 usage example → BodyM Datasetcomputer visiondeep learning The first large public body measurement dataset including 8978 frontal and lateral silhouettes for 2505 real subjects, paired with height, weight and 14 body measurements. The following artifacts are made available for each subject.
The data is split into 3 sets - Training, Test Set A, Test Set B. For the training and Test-A sets, subjects are photographed and 3D-scanned by in a lab by technicians. For the Test-B set, subjects ... Details → Usage examples
See 1 usage example → Google Brain Genomics Sequencing Dataset for Benchmarking and Developmentbioinformaticsfastqgeneticgenomiclife scienceslong read sequencingshort read sequencingwhole exome sequencingwhole genome sequencing To facilitate benchmarking and development, the Google Brain group has sequenced 9 human samples covering the Genome in a Bottle truth sets on different sequencing instruments, sequencing modalities (Illumina short read and Pacific BioSciences long read), sample preparation protocols, and for whole genome and whole exome capture. The original source of these data are gs://google-brain-genomics-public. Details → Usage examples
See 1 usage example → Humor patterns used for querying Alexa trafficamazon.sciencedialogmachine learningnatural language processing Humor patterns used for quering Alexa traffic when creating the taxonomy described in the paper "“Alexa, Do You Want to Build a Snowman?” Characterizing Playful Requests to Conversational Agents" by Shani C., Libov A., Tolmach S., Lewin-Eytan L., Maarek Y., and Shahaf D. (CHI LBW 2022). These patterns corrospond to the researchers' hypotheses regarding what humor types are likely to appear in Alexa traffic. These patterns were used for querying Alexa traffic to evaluate these hypotheses. Details → Usage examples
See 1 usage example → MODIS MYD13A1, MOD13A1, MYD11A1, MOD11A1, MCD43A4agriculturedisaster responsegeospatialnatural resourcesatellite imagerysustainability Data from the Moderate Resolution Imaging Spectroradiometer (MODIS), managed by the U.S. Geological Survey and NASA. Five products are included: MCD43A4 (MODIS/Terra and Aqua Nadir BRDF-Adjusted Reflectance Daily L3 Global 500 m SIN Grid), MOD11A1 (MODIS/Terra Land Surface Temperature/Emissivity Daily L3 Global 1 km SIN Grid), MYD11A1 (MODIS/Aqua Land Surface Temperature/Emissivity Daily L3 Global 1 km SIN Grid), MOD13A1 (MODIS/Terra Vegetation Indices 16-Day L3 Global 500 m SIN Grid), and MYD13A1 (MODIS/Aqua Vegetation Indices 16-Day L3 Global 500 m SIN Grid). MCD43A4 has global coverage, all... Details → Usage examples
See 1 usage example → Orcasound - bioacoustic data for marine conservationbiodiversitybiologycoastalconservationdeep learningecosystemsenvironmentalgeospatiallabeledmachine learningmappingoceansopen source softwaresignal processing Live-streamed and archived audio data (~2018-present) from underwater microphones (hydrophones) containing marine biological signals as well as ambient ocean noise. Hydrophone placement and passive acoustic monitoring effort prioritizes detection of orca sounds (calls, clicks, whistles) and potentially harmful noise. Geographic focus is on the US/Canada critical habitat of Southern Resident killer whales (northern CA to central BC) with initial focus on inland waters of WA. In addition to the raw lossy or lossless compressed data, we provide a growing archive of annotated bioacoustic bouts. Details → Usage examples
See 1 usage example → PersonPath22computer vision PersonPath22 is a large-scale multi-person tracking dataset containing 236 videos captured mostly from static-mounted cameras, collected from sources where we were given the rights to redistribute the content and participants have given explicit consent. Each video has ground-truth annotations including both bounding boxes and tracklet-ids for all the persons in each frame. Details → Usage examples
See 1 usage example → Pre- and post-purchase product questionsamazon.sciencemachine learningnatural language processing This dataset provides product related questions, including their textual content and gap, in hours, between purchase and posting time. Each question is also associated with related product details, including its id and title. Details → Usage examples
See 1 usage example → The Multilingual Amazon Reviews Corpusmachine learningnatural language processing We present a collection of Amazon reviews specifically designed to aid research in multilingual text classification. The dataset contains reviews in English, Japanese, German, French, Chinese and Spanish, collected between November 1, 2015 and November 1, 2019. Each record in the dataset contains the review text, the review title, the star rating, an anonymized reviewer ID, an anonymized product ID and the coarse-grained product category (e.g. 'books', 'appliances', etc.) Details → Usage examples
See 1 usage example → WikiSum: Coherent Summarization Dataset for Efficient Human-Evaluationamazon.sciencemachine learningnatural language processing This dataset provides how-to articles from wikihow.com and their summaries, written as a coherent paragraph. The dataset itself is available at wikisum.zip, and contains the article, the summary, the wikihow url, and an official fold (train, val, or test). In addition, human evaluation results are available at wikisum-human-eval... Details → Usage examples
See 1 usage example → Wizard of Tasksamazon.scienceconversation datadialogmachine learningnatural language processing Wizard of Tasks (WoT) is a dataset containing conversations for Conversational Task Assistants (CTAs). A CTA is a conversational agent whose goal is to help humans to perform real-world tasks. A CTA can help in exploring available tasks, answering task-specific questions and guiding users through step-by-step instructions. WoT contains about 550 conversations with ~18,000 utterances in two domains, i.e., Cooking and Home Improvement. Details → Usage examples
See 1 usage example → Airborne Object Tracking Datasetamazon.sciencecomputer visiondeep learningmachine learning Airborne Object Tracking (AOT) is a collection of 4,943 flight sequences of around 120 seconds each, collected at 10 Hz in diverse conditions. There are 5.9M+ images and 3.3M+ 2D annotations of airborne objects in the sequences. There are 3,306,350 frames without labels as they contain no airborne objects. For images with labels, there are on average 1.3 labels per image. All airborne objects in the dataset are labelled. Details →
Amazon Berkeley Objects Datasetamazon.sciencecomputer visiondeep learninginformation retrievalmachine learningmachine translation Amazon Berkeley Objects (ABO) is a collection of 147,702 product listings with multilingual metadata and 398,212 unique catalog images. 8,222 listings come with turntable photography (also referred as "spin" or "360º-View" images), as sequences of 24 or 72 images, for a total of 586,584 images in 8,209 unique sequences. For 7,953 products, the collection also provides high-quality 3d models, as glTF 2.0 files. Details → MWIS VR Instancesamazon.sciencegraphtraffictransportation Large-scale node-weighted conflict graphs for maximum weight independent set solvers Details → Registry of Open Data on AWSjsonmetadata The Registry of Open Data on AWS contains publicly available datasets that are available for access from AWS resources. Note that datasets in this registry are available via AWS resources, but they are not provided by AWS; these datasets are owned and maintained by a variety of government organizations, researchers, businesses, and individuals. This dataset contains derived forms of the data in https://github.com/awslabs/open-data-registry that have been transformed for ease of use with machine interfaces. Curren... Details → TSBenchbenchmarkdeep learningmachine learningmeta learningtime series forecasting TSBench comprises thousands of benchmark evaluations for time series forecasting methods. It provides various metrics (i.e. measures of accuracy, latency, number of model parameters, ...) of 13 time series forecasting methods across 44 heterogeneous datasets. Time series forecasting methods include both classical and deep learning methods while several hyperparameters settings are evaluated for the deep learning methods.In addition to the tabular data providing the metrics, TSBench includes the probabilistic forecasts of all evaluated methods for all 44 datasets. While the tabular data is smal... Details → The Klarna Product-Page Datasetcommercecomputer visiondeep learninggraphinformation retrievalinternetmachine learningnatural language processing A collection of 51,701 product pages from 8175 e-commerce websites across 8 markets (US, GB, SE, NL, FI, NO, DE, AT) with 5 manually labelled elements, specifically, the product price, name and image, add-to-cart and go-to-cart buttons. The dataset was collected between 2018 and 2019 and is made availalbe has MHTML and as WebTraversalLibrary-format snapshots. Details → Which of the following are usually good data source?Which of the following are usually good data sources? Select all that apply. Vetted public datasets, academic papers, and governmental agency data are usually good data sources.
What are the main benefits of open data select all that apply?What are the main benefits of open data? Open data restricts data access to certain groups of people. Open data increases the amount of data available for purchase. Open data makes good data more widely available.
Which of the following are types of data bias often encountered in data analytics select all that apply?Correct. Observer bias, interpretation bias, and confirmation bias are types of bias often encountered in data analytics.
What is the process for arranging data into a meaningful order to make it easier to understand analyze and visualize?Data sorting is any process that involves arranging data into some meaningful order to make it easier to understand, analyze, or visualize. When working with data, sorting is a common method used for visualizing data in a form that makes it easier to digest the story you want to tell with the data.
|
