Red flags to look out for when joining a data team

https://eugeneyan.com/writing/red-flags/

Data: No data, or data that’s poorly organized and/or inaccessible.

Without basic data infra in place, a data scientist would face an uphill battle trying to contribute value via analytics or machine learning. Most will likely be frustrated with the grind of data acquisition, organization, cataloging, and building pipelines.

A few questions to suss out this red flag:

• What data is being generated or collected by your systems?
• What are the key objects in your data, such as customers, items, or transactions, and what is the approximate number of new rows daily or monthly?
• As a new joiner, how would I access the data?

Roadmap: No/poor plan on how the team will deliver value to customers and the business.

When lean times come, teams that don’t contribute to the business are likely first to be cut.

Questions to ask about the roadmap:

• How does the team deliver value to customers and the business?
• What is your roadmap and planned deliverables for the quarter? And the year?
• How does success look like for the team? What are the team’s KPIs?

Role: Misaligned expectations on the role and career progression guidelines.

Questions to ask about the role and progression:

• What should someone in this role deliver in the first 100 days? And the first year?
• Of the most effective people in this role, what key skills and behaviors do they have?
• How is success in this role defined and measured?
• What does someone have to demonstrate to progress to the next level?

Manager: Incompetent, lacks vision, abusive, etc

One way is to ask the hiring manager to share the contact details of 2 - 3 people who used to work with them, people who can speak freely without fear of direct repercussion. Alternatively, speak to the most tenured person on the team.

Questions to probe for managerial red flags:

• What are the strengths and weaknesses of the manager?
• Did you enjoy working with the manager? Why or why not?
• How did the manager support the team and help them grow?
• Would/did you enjoy hanging out with the manager in a casual setting?

Tooling: They use outdated or proprietary tools that are barely transferable to other roles.

Some questions on the tooling:

• What key tools does the team use in their day-to-day?
• What is your tech stack?

Org structure: The data science team rolls up to an unusual C-level

While there isn’t a definitive C-level for the data team to report to, I’ve seen organizations where data team rolled up to CTO or CIO or VP Eng that seem to work well. If the org has a CDO, that would be ideal. I would be concerned if data team reported to the CFO or CMO.

Questions to ask on org structure:

• Who does the data science team roll up to?
• Who are the main stakeholders of the data science team?

Iteration speed: The team moves too fast/slow for your liking.

I’ve been on teams where you can deploy a new model in minutes and run an AB test every two weeks, and I’ve been on teams where deployment meant several people nursing the pipeline overnight, with little to no AB testing throughout the year. The former allows rapid experimentation, learning, and growth, while the latter leads to friction, sluggishness, and stagnation.

Questions to get a sense of the team’s iteration rate:

• What was the last thing the team shipped? How long did that take?
• How many AB tests did the team conduct in the last month or quarter? What were the outcomes?