Feat/data models estimation services

[shreya-m9]

Overview

Added pydantic validation models for data ingestion to the models folder and processing services schemas into the services folder. All files in those folders were added/edited.

Technical Description

We achieved the ticket's asks of defining data models through pydantically validated schemas that:

1. list expected columns for each row in our database (vehicle state model), the raw and cleaned sensor data (sensor model), and metrics calculated for a time window (metrics model)
2. outline the services needed for each model

How Has This Been Tested?

Created tests for each model, in their own files within tests folder. Mainly tests whether a variety of inputs can be handled in the models since no logic has been implemented in the services

Ticket Review

Answer the following:

1. What went well

• Collaboration was great, we discussed every step of the project. We went over our code multiple and peer reviewed each other.

3. What was difficult

• Understanding what the services/models needed to entail

5. If you could start over what would you do differently

• Maybe we could have spread out this ticket's work over a couple of days

[henrybednarz]

We gotta use numpy for matrix operations otherwise shit gonna get wicked

[henrybednarz]

The stddev should probably exist as a mapping instead of copying the value to every data point. If we have ~100 different sensors and like ~10million sensor readings then we are storing a lot of duplicate values.

[henrybednarz]

The services and models all look good. Here are my other thoughts which I also had for other peoples tickets:

I think for non-end events it does not make sense to use pydantic models. This is on me for not thinking hard at the beginning. What is happening there is a ton of memory/computation overhead to use python lists for this type of data (especially with matrix operations). So we need to shift to using numpy arrays/pandas. I think we should discuss as a group at the next meeting to decide how we should move forward. The good news is all the work is basically the same just changing it from pydantic to a numpy/pandas df. Good work though the models all look great.