Elizabete Kalnozola Manager   •   about 3 years ago

General Problem Statement

Prize - Vertex Ventures Internship for Working Adult, Fresh Graduate & Undergraduate in Singapore

Schools are getting more data driven, we have such a school that has been collecting data across years to understand results, enrollments across states and districts.

They would like to consolidate all the data they have across years and understand how their report cards or results have changed over the years. They understand the benefit of the cloud, and create a cloud based system to help them with their needs.

The challenge is to help build a scalable and cost optimized solution to help the school with the following:

Ingest multiple years data, ensure to select the storage format that helps with good compression of the data. The development team at the school would prefer low code options . Design the ingestion to support new year’s data to come in without having to change code.
The teams are willing to adapt CI/CD, how can that be made possible with this solution?
The end - result is to create dashboards showcasing trends across years, any exploratory analysis that can be highlighted from the data will help the schools to fine tune their policies. Think about the serving layer for the dashboards, ensure it is cost-effective and scalable, the team's inclination towards the cloud is because they know it is a pay as go model, PaaS services that can be paused when not in use.
The school is very concerned about security; how can security be embedded across the solution?
Please also share the performance and pricing details for the solution.
Dataset: Downloads | NYSED Data Site // Enrollment Database - https://data.nysed.gov/downloads.php

Hint: Azure Synapse, Power BI