First Impressions of Mage: A Promising Tool for Data Engineering

Exploring the strengths and weaknesses of Mage based on my initial experience with the tool.

My first impressions while testing the Mage tool were very positive.

For my project, I used the Marvel API to read data, performed some transformations, and saved the results in an AWS bucket and locally in Postgresql.

Strengths:

Installation is easy, and it’s possible to start using the tool quickly. Despite having many features, the user interface is simple and easy to use.

The handling of DAGs is intuitive and flexible, which greatly assists in development. The ability to execute the blocks above helps speed up the development process.

One of the things I liked most was the integration with tests within each block of code, which helps ensure that the code is working correctly.

Data visualization integrated into the user interface is a nice touch.

Weaknesses:

While it’s interesting to develop using the web interface, some IDE features are missed.

Sometimes, hitting the correct scrollbar can be a bit tricky due to the proximity of the development area and side view scrollbars.

The documentation still needs to be improved, especially in terms of practical examples of some of its functions. (Great opportunity for those who want to contribute to the community.)

Final thoughts:

Overall, I’m very excited about the Mage tool. It has unique strengths that address important points in the development of a data engineer. I intend to continue following this project with great anticipation.

Check my project on GitHub: https://github.com/deniswoliveira/mage-ai-marvel-project/