Welcome to the next entry in my series where I’ll be chronicling the journey of building a new platform, week by week. This project is not just about the technical aspects of platform development but also about the business insights, stakeholder interactions, and the tools that make it all possible. I invite you to my world to come along for the ride.
I recently joined Gallo Mechanical as their VP of Analytics. Gallo is a national leader in the construction services industry and from day one of speaking to them, I knew this was a place I wanted to be. The success of Gallo is a testament to the strength and focus of its leadership and employees. Im excited to help bring a data and analytics mindset and apply it to the day to day and strategic operations of the business.
Let’s dive into Weeks 2 and 3.
Setting the Stage
Ongoing business discovery
Continuing on the path to understand the business in greater detail. The construction services industry has a deceptively complex ecosystem to how business gets done. There are so many opportunities for data to be effective. The last 2 weeks (Thanksgiving was a short week) have let me gain a tremendous amount of knowledge. The most important thing you can do is truly understand the business. This will allow you to accelerate your data foundation work.
Vendor discussions for access to data
One of the big items to do is to get access to data. We are using snowflake (still in trial). Ive talked to Fivetran and Portable.io for bringing data in. At Shiftkey, we had internal MySQL on AWS and it was pretty simple to turn on logging and change tracking and ingest the data into the warehouse. With Gallo, we have over 5 distinct software platforms that we need to connect to for pulling down data.
I had to meet and establish a service contract for API access so that we could pull it down via Portable. Ethan and the team over at Portable are pretty awesome. They already had one of the connectors we needed and quickly created the second.
We already have our fleet management platform pushing 100% of its data into the raw, unstructured warehouse as well as 30 of our core platform (Viewpoint VISTA) APIs.
Modeling and Planning
Business Model
Using MIRO, I have started to model our business structure into a visual workflow so I can understand the path a job takes. We do everything from small service related work to large, enterprise projects spanning months or even years. It is an amazing company and to watch them do what they do is truly inspiring.
Architecture
As we look to build our structured data platform, I have sketched out multiple versions of how the data will work. We have three main companies and each will require some flavor of its own so instead of getting too overwhelmed, I decided it is best to focus on the core of our business. If you do this, you will be able to get to business value quickly.
This is also where you identify potential issues with data integrity. Look at the tables and ask:
- Are people entering all the data that is needed? (missing fields, null values, etc)
- Can you find areas of improvement for the business for capturing data?
Focus
Once our fleet data was ingested, I spent a few hours just exploring the data to see what was there and see if I could quickly build a set of entities around our fleet management (drivers, vehicles, trips, etc). I created a few entity and aggregation tables in the warehouse. This is also where I developed our naming conventions for our tables to make sure there was some semantic sense to it.
I used a low-code tool called make.com to build my first data workflows while I work on deciding what tool I should use for ETL. Considering dbt but trying to make sure I focus on quick wins without long tail implementation projects. I am still a party of one here so some decisions have to be made that include a little technical debt.
Action Summary
- Ingested data via portable to the raw warehouse
- Designed the core data structures that I have access to
- Explore data and wrapped my head around how this data matches the business expectations.
- Met with business leaders and saw that there are mini-apps built on excel everywhere. This will present a challenge.
Looking Ahead
As we move into Week 4, the focus will shift towards ingesting more data and continuing to build some models. I really want to get some basic HEX reports out by Christmas so I can prove out the benefits of data here at Gallo. I think there will also be some additional prioritization once the data from VISTA is there.
Week 1 – Laying the foundation
Join me next week as we delve deeper into the intricacies of platform development and witness how a concept gradually transforms into a tangible, functioning entity. Your thoughts and suggestions are always welcome, so feel free to email me your comments. john@slingspace.com
