Week 3

Cloud Computing, APIs & Data Governance

Introduction

Agricultural data are increasingly collected, stored, and processed across cloud platforms and connected systems. This week we explore the infrastructure and governance needed to work with these data responsibly.

Learning goals

Understand basic concepts of cloud computing, APIs, and data infrastructure
Connect Git workflows with cloud-based computing
Understand data governance and data lineage
Recognize the importance of data agreements and responsible data use

Lecture - Cloud Computing, APIs & Hardware

Introduction to cloud computing, APIs, hardware, compute and storage, and cloud-based agricultural data workflows.

Topics include Databricks and Git integration (creating a notebook, reading the data and making a basic data profile), data agreements and licensing.

Lab - Cloud-Based Data Workflow

Hands-on introduction to a simple cloud-based workflow using Git and Databricks.

Discussion - Data Governance & Data Lineage

How do we know where agricultural data come from, how they have been changed, and who can use them?

Introduction to data governance and tracking data lineage in agricultural datasets.

Assignment

Get Databricks Education access as a student. Complete the student test/tutorial. Invite Dr. Miel to your Databricks workspace.
Create or use your personal GitHub repository and make sure the repository is set to PUBLIC.
Complete the simple cloud-based workflow assignment using Git and Databricks. Submit the link to your GitHub page in Canvas.
Select an open-source license by following the provided flowchart. Add the selected license to your personal GitHub repository and submit the GitHub link in Canvas.
Complete the Data Agreement Template. Upload the completed template to your personal GitHub repository and submit the GitHub link in Canvas.

Looking ahead

Next week we will introduce machine learning and artificial intelligence and begin connecting our data workflows to predictive models.

Back to course schedule ⏎