Where will your project live?

From my experience working with organizations from a variety of industries, it is not uncommon for people to choose the technology or tool they want to use first. I think this is a bad idea. To reinforce that idea, I specifically put this as the last set of questions when refining your project.

Technologies should fit you and your project

I think this is one of the more challenging aspects of any project. It is not only, "where is this happening", but also "how are we going to complete this?". Given this, it is easy to get convinced that one tool is better than any other.

We need to remember who the project is intended for, and their entire skillset. Vendors may offer you a great deal on a new product, but be cautious that you may be picking a new tool that no one knows how to use.

Understanding your team’s skills is important for having a successful implementation. The trap of purchasing something that no one knows how to use is a common pitfall for many projects. A common, but avoidable pitfall. Avoid this by understanding the full scope of the project and the team who will be running the implementation.

Avoid more tools

If you can, try to avoid adding another tool as it can add to complexity. For example, if your company already has a data warehouse solution, such as Snowflake or Databricks, find out if you are able to gain access! Try to avoid adding more tools and complicating your tech stack.

If someone approaches you wanting to switch technologies, listen to their the reasons why they want to switch. While their reasons for switching may be valid be sure they aren't just because they are not familiar with the existing technologies or just like another tool. Be open to switching or adding, but this is important to lay out the timeline differences for supporting this shift or addition.

Picking the right tech stack

Lets be honest, you will probably question your choice and may end up switching your stack down the road. Someone once told me before I started a new job, "don't be surprised if you want to re-write all of your work in 6 months". I wouldn't recommend re-creating something just because, but it is important to keep in mind that tool selection can be challenging and you make a mistake. Conduct as much as research as you can, but nothing compares to actually working with a technology.

I believe three key points will help you avoid the potential regret of your technology selection.

1. Understand your capabilities and how that will affect your ability to use these tools. If you don't have experience deploying new infrastructure, it is best to stay away from the free open-sourced version and lean towards a managed service.

2. Think about the current state of your business and where you think you will be in the future. Some tools are great for smaller workloads, while others are better for large scale workloads. Find the tool that works for your situation, and try to avoid switching in the future.

3. How will your entire tech stack interact? Many tools are built to work with others and make it so you can easily integrate your technologies, this capability will help you to quickly scale your business. Avoid tools that don’t natively integrate with your stack, this can cause an unnecessary burden on you and will cause you headaches in the future.

I started this series off by exploring initial questions you should be asking before starting any project. Refining your project is key to long-term success. Check out our list of questions for project refinement, and leave comments below of questions that you think should be added!

Share this with someone who is working on an analytics project! As a token of my appreciation, you will receive a free questionnaire to help you refine your next project!

Are you having trouble determining the timeline of your project? Need help differentiating between real-time data pipelines and batch loads? Reach out as I'd like to learn more about how I can help!