Data Science vs Data Engineering: Which is right for you, and why?

For the last decade, the data science job market has seen some stunning growth and consistently high salaries. Increasingly, however, there’s an alternative emerging for someone interested in implementing data solutions: data engineering. According to Dice and Burning Glass’ NOVA platform, in April 2019 “data engineer” was the top tech job, seeing 88% growth year-over-year, compared to 49% YoY growth for “data scientist.” While more recent numbers in LinkedIn’s 2020 US Emerging Jobs report show more modest growth, they’re still quite bullish, with the two roles together seeing 35% growth.

Though data science jobs are on balance better compensated, there’s also not much daylight here: according to Salary.com, data scientists in the US usually earn between $124,770 and $154,336, while data engineers’ salaries typically fall between $98,287 and $130,038 — considerable overlap.

Given the sunny job outlook and substantial compensation for both jobs, the decision as to which to pursue ultimately comes down to which interests you most. But if you’ve just started learning about data-centric careers, making this decision can be more difficult than it sounds. To help, in this article we’ll break down what each discipline is, the skills practitioners of each need to have, and how to conceptualize the difference between the two, plus give some tips for how to come to a decision and what to do once you do.

What is data science?

Data science leverages computer science, applied mathematics, machine learning, and data management to extract insights from data and build new techniques and tools for doing so. Insights produced by data science can:

support business decision-making, such as whether to enter a new market at Lululemon,
catalyze research and development (R&D), such as developing new metrics for social analytics at Facebook,
or even form the basis of new products, services, and marketing initiatives, such as Spotify’s annual Wrapped campaign.

What skills does a data scientist need to have?

To succeed in their role, a data scientist needs to possess skills in the following areas:

Computer science, including fluency with programming languages like SQL, Python, and R; software design and engineering

Applied mathematics, including statistics, probability, and linear algebra

Machine learning, including unsupervised learning, supervised learning, reinforcement learning, and deep learning with neural networks, as well as familiarity with machine learning libraries like Pandas

Data management, including data wrangling; data mining; database maintenance, querying, and manipulation; data cleansing; big data processing

Data visualization, including tools like matplotlib and Tableau

What does a data scientist’s day-to-day look like?

Though it can vary depending on their seniority and the company they’re working at, a data scientist’s day-to-day will typically consist of developing and advancing analytics projects, including coding and training machine learning models, interpreting results of analysis, and putting together compelling presentations to share insights with internal and external stakeholders.

At Meta, a data scientist working on Reality Labs sales analytics would have the following responsibilities and qualifications:

Data Scientist, Reality Labs Sales Analytics — Meta

Responsibilities:

Lead analytics projects end-to-end in partnership with Product, Engineering, and cross-functional teams to inform, influence, support, and execute product strategy and investment decisions
Influence product direction through clear and compelling presentations to leadership
Work with large and complex data sets to solve a wide array of challenging problems using different analytical and statistical approaches
Apply technical expertise with quantitative analysis, experimentation, data mining, and the presentation of data to develop strategies for our products that serve billions of people and hundreds of millions of businesses
Identify and measure success of product efforts through goal setting, forecasting, and monitoring of key product metrics to understand trends
Define, understand, and test opportunities and levers to improve the product, and drive roadmaps through your insights and recommendations

Qualifications:

Bachelor's degree in Mathematics, Statistics, related technical field, or equivalent practical experience
A minimum of 4 years of work experience (2 years with a Ph.D.) in applied analytics
Experience with data querying languages (e.g. SQL), scripting languages (e.g. Python), and/or statistical/mathematical software (e.g. R)

For a deeper dive into what it’s like to be a data scientist, check out a day in the life of this Amazon data scientist:

What is data engineering?

Data engineering is focused on the implementation, evaluation, and maintenance of data architectures such as data pipelines, databases, and other data processing systems. Data engineering is essential for tasks like:

optimizing a large internet retailer’s customer relationship management (CRM) database,
providing efficient access to big data sets to data scientists’ machine learning projects at social media giants,
or ensuring reliable pipelines of data from sensors and actuators for automated industrial manufacturing.

What skills does a date engineer need to have?

The skills needed to be a data engineer overlap substantially with those needed for data science, but in general data engineers rely more heavily on computer science and data management skills:

Computer science, including extensive knowledge of programming languages like SQL, Python, R, C and C++, and Java; knowledge of operating systems including Linux, Unix, Windows, and macOS; software development

Data management, including tools for database management, data transformation, data mining, data pipelines, and cloud computing

Basic data analytics, including statistical analysis and artificial intelligence and machine learning basics

Data visualization, including tools like matplotlib and Tableau

What does a data engineer’s day-to-day look like?

A data engineer’s day-to-day is coding heavy — they’ll often spend hours writing data pipelines and testing and debugging shipped systems — but they often spend a significant amount of time in meetings. After all, to develop effective and efficient data solutions, data engineers need to communicate constantly with the users of those pipelines to ensure that the solutions are meeting user needs.

Returning to Meta, a data engineer working in the analytics program would have the following responsibilities and qualifications:

Data Engineer, Analytics — Meta

Responsibilities:

Conceptualize and own the data architecture for multiple large-scale projects, while evaluating design and operational cost-benefit tradeoffs within systems
Create and contribute to frameworks that improve the efficacy of logging data, while working with data infrastructure to triage issues and resolve
Collaborate with engineers, product managers, and data scientists to understand data needs, representing key data insights in a meaningful way
Define and manage SLA for all data sets in allocated areas of ownership
Determine and implement the security model based on privacy requirements, confirm safeguards are followed, address data quality issues, and evolve governance processes within allocated areas of ownership
Design, build, and launch collections of sophisticated data models and visualizations that support multiple use cases across different products or domains
Solve our most challenging data integration problems, utilizing optimal ETL patterns, frameworks, query techniques, sourcing from structured and unstructured data sources
Assist in owning existing processes running in production, optimizing complex code through advanced algorithmic concepts
Optimize pipelines, dashboards, frameworks, and systems to facilitate easier development of data artifacts
Influence product and cross-functional teams to identify data opportunities to drive impact
Mentor team members by giving/receiving actionable feedback

Qualifications:

Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience.
4+ years of work experience in data engineering (a minimum of 2+ years with a Ph.D)
Experience with SQL, ETL, data modeling, and at least one programming language (e.g., Python, C++, C#, Scala, etc.)

Preferred Qualifications:

Master's or Ph.D degree in a STEM field
Experience with one or more of the following: data processing automation, data quality, data warehousing, data governance, business intelligence, data visualization, data privacy
Experience working with terabyte to petabyte scale data

For more on what a data engineer’s day-to-day looks like, check out this video from a Liaison Technologies data engineer:

What’s the difference between data science and data engineering?

From the descriptions of the disciplines, the necessary skills, the job descriptions, and the day-in-the-life videos above, we gain an understanding of data science and data engineering as complicated, complimentary, and ultimately essential disciplines as part of a company’s larger data operations.

While data science provides new methods for developing crucial insights from data, data engineering ensures that a company’s data operations run smoothly, including ensuring that data is available and prepared for whatever needs data scientists may have.

How do you know which is right for you?

As you are deciding between these career paths, keep in mind the skills required and what your day-to-day would look like for each. If you’re interested in spending hours writing pipelines and collaborating with product managers, data scientists, and other internal stakeholders to ensure that their data needs are met, data engineering might be the right path for you. If instead you’re interested in ideating new questions and how to answer them to further business goals, and then presenting your answers to relevant stakeholders through data storytelling, data science might be the preferable path.

Bonus: What’s a data architect?

But wait, there’s more: before you come to a decision, you should also consider data architecture. While data engineers focus primarily on developing — that is, building — data architectures, only in some cases designing them, data architects focus solely on the design of these architectures. Data architects require a similar skillset as a data engineer, heavy on computer science and data management, but they also need to have a design mindset in order to identify problems, ideate solutions, prototype the best of these, and then iterate to improve their design over time. For this, the average data architect earns an average of $125,898 in the US.

Is a data architect career right for you?

If you’re interested in overseeing engineering projects and implementing design thinking, data architecture might be a great career path for you. That said, it can sometimes be difficult to jump immediately into data architecture without gaining experience first as a data engineer or data scientist. After all, as a data architect, you are often designing solutions that data engineers will implement.

What’s next?

You might think that you need to pick one of these paths — data science, data engineering, or data architecture — and stick to it, but as we’ve just demonstrated, this isn’t really the case. Often, you can pursue further education in one discipline and still preserve optionality to pivot into another discipline down the road. But you still have a decision to make; depending on your background, there are different educational paths you can take. Here’s how to learn more:

If you’re still in high school and have the time and money to pursue a four-year degree, a bachelor’s in computer science or applied mathematics will get you started building the necessary skillset for your data career. See our guide to data science and data analytics programs for more.

If you’re looking for an accelerated path into an entry-level position — perhaps you’re considering transitioning careers — check out our guide to data science bootcamps. These combine comprehensive data science instruction with career services to place students in entry-level data analyst, data scientist, and data engineer roles. While bootcamps can be pricey — the average bootcamp cost $11,727 in 2020 — they are still less expensive than traditional degree programs, and some even offer payment plans, income-share plans, or money-back guarantees.

If you have a bachelor’s degree, are looking to advance or transition your career, and are comfortable spending time and money on a two-year degree, see our guide to data analytics and data science master’s programs. A master’s degree generally comes with a boost in compensation once you find a job. In fact, some senior positions require that candidates have at least a master’s degree, and in some cases a Ph.D.

If you’re interested in pursuing a master’s degree but don’t want to take time off of work, see our guides to online data analytics and data science master’s programs. Online programs often provide flexibility in how and when you study, so that you can continue working and taking care of other obligations. This, combined with not having to relocate or commute, can provide a cost savings that for many can make the financial calculus of graduate study worth it.

If you already have certain relevant skills and are looking to expand your skillset to pursue jobs in data science, data engineering, or data architecture, data science certificate programs might offer the most return on investment. These vary in price to fit almost any budget, from free massive open online courses (MOOCs) to live online university professional and graduate certificates costing several thousands of dollars. As with online master’s degrees, they are frequently flexible, allowing you to continue to work while you complete them.

If you still want to learn more before diving into the educational options available to you, check out the following articles: