Great blog. Happy to see that you still keep Scala in your courses. If we wanted to touch OSS projects, or work with legacy projects, one needs to work with Scala. Would be interested to know what could be the take if any newer data platforms are considering Scala at all ?
The point about AI/data integrations, especially vector databases and RAG pipelines, really resonated with me. This roadmap provides invaluable clarity in a rapidly evolving space. While tool mastery is critical, I also belive a strong grasp of data ethics and responsible AI practices will become increasingly paramount for future data engineers.
Great blog. Happy to see that you still keep Scala in your courses. If we wanted to touch OSS projects, or work with legacy projects, one needs to work with Scala. Would be interested to know what could be the take if any newer data platforms are considering Scala at all ?
Thanks for this 💐🙏
The point about AI/data integrations, especially vector databases and RAG pipelines, really resonated with me. This roadmap provides invaluable clarity in a rapidly evolving space. While tool mastery is critical, I also belive a strong grasp of data ethics and responsible AI practices will become increasingly paramount for future data engineers.
If you want to start using AI agents for DQ and RCA ... , check out this article from Anthropic: https://www.anthropic.com/engineering/building-effective-agents ... also you can check: https://eugenede.substack.com/p/integrating-ai-into-data-engineering ...
Tysm!