As artificial intelligence (AI) moves deeper into data engineering, much of the public conversation points to a single outcome: software will write the pipelines, and the engineers behind them will matter less. Sowmya Yattapu, a principal data engineer with 15 years in financial services, has spent the past several years examining that claim, and her published work makes a more measured case.
In a paper for the International Journal of Scientific Research and Engineering Trends (IJSRET), Yattapu set out how AI is reshaping data engineering across databases, warehousing, and analytics. Her argument is that AI is absorbing the mechanical layer of the work, the repetitive building and fixing, while the part that defines a capable engineer moves toward design, judgment, and oversight. The role, in her account, is being rewritten rather than retired.
The view carries weight because it comes from practice. As a principal engineer responsible for the reliability of large financial data systems, Yattapu writes from inside the problem rather than about it. She has built her career in a setting where the margin for error is thin: systems that move transactions, balances, and regulatory reports, where a single wrong number can travel a long way before anyone catches it.
What automation should own
One example from her own work illustrates how she applies the principle she advocates. In production systems, a recurring class of errors known as data discards once required manual correction each time they surfaced, a slow process whose results varied depending on who was on call. Yattapu built an automated response that detects and resolves these errors without human intervention, applying the same fix consistently and in one go. In a domain where inconsistent remediation can corrupt downstream records, removing the person from that loop did more than save time. It made the outcome dependable by design.
That is the kind of work she believes automation should own: the repetitive, well-understood tasks that are safer without a human in the middle.
The judgment that stays human
The harder questions sit above it. What should a system do, and what must it protect against? How much resilience is a given component actually worth? In finance, these are not abstract choices. A bad data feed or a job that runs on incomplete numbers can reach reports, compliance filings, and customer balances before anyone notices.
Those judgments, in Yattapu’s framework, stay with the engineer accountable for the result. A model can propose a pipeline. It cannot weigh what a particular failure would cost or decide where an added safeguard would earn its expense. The discipline, as she describes it, is separating into two layers: a mechanical one that automation handles well, and a layer of judgment that grows more important as the first is handed off.
That distinction sets her apart from a common instinct in the industry, which treats automation as an end in itself. Her position is that automation is only as good as the decisions that frame it. A system that runs without oversight is efficient until the moment it does something no one intended, and in financial data, that moment is costly. Her work makes the case for building automation and accountability together, rather than trading one for the other.
She sees a practical lesson in that split for the engineers coming up behind her. Fluency in writing code counts for less when a model can draft it. Knowing which system to build, what it must never get wrong, and how it should fail safely counts for more. On her reading, AI raises both the floor and the ceiling of the work at once: routine engineering becomes faster and cheaper, while the demands on senior judgment rise. The engineers who stay essential will be the ones who can carry that judgment.
That mix of hands-on engineering and published analysis is what gives her argument its weight. For an industry investing heavily in automation, her case amounts to a useful correction: automate the right things, and keep human judgment on the decisions that carry real cost.
