Hadoop Developer

Vintti is a staffing agency that leverages the geographical advantage of Latin America to benefit US businesses. We connect companies with professionals who work in time zones closely aligned with or identical to US hours, ensuring seamless communication and collaboration. This synchronicity allows for real-time interaction, enhancing productivity and eliminating the delays often associated with offshore staffing.

A Hadoop Developer is a specialized software professional responsible for the design, development, and deployment of applications within the Hadoop ecosystem. Their key focus is on managing and processing vast amounts of data, utilizing tools like MapReduce, Hive, Pig, and HBase. These developers optimize the performance of data workflows, ensuring efficient storage, retrieval, and analysis. They collaborate closely with data scientists, analysts, and other IT professionals to transform complex data into actionable insights, supporting business decision-making processes and driving strategic initiatives within an organization.

- Bachelor's degree in Computer Science, Information Technology, or related field
- Proven experience as a Hadoop Developer or similar role
- Proficiency in Hadoop-related technologies: HDFS, MapReduce, Yarn, Pig, Hive, HBase, Flume, and ZooKeeper
- Strong programming skills in Java, Python, or Scala
- Experience with data warehousing and ETL tools
- Familiarity with data modeling, data architecture, and relational databases
- Hands-on experience with big data tools like Apache Spark, Kafka, or Apache Nifi
- Knowledge of Linux operating system and bash/shell scripting
- Proficiency in SQL for querying and managing relational databases
- Experience with version control systems, such as Git
- Strong problem-solving and analytical skills
- Knowledge of best practices in data security and privacy
- Familiarity with Agile development methodologies and tools like JIRA
- Strong written and verbal communication skills
- Ability to work both independently and in a collaborative environment
- Experience with cloud platforms like AWS, Google Cloud, or Azure
- Certification in Hadoop or related technologies (preferred)
- Excellent organizational and time management skills

- Develop and maintain Hadoop jobs and workflows
- Design and implement scalable data pipelines
- Perform ETL processes for data ingestion and transformation
- Monitor and optimize Hadoop cluster performance
- Manage cluster resources and configure for high availability
- Create custom data processing applications using MapReduce, Hive, and Pig
- Collaborate with data teams to integrate data models and algorithms
- Conduct data validation and ensure data quality
- Document data workflows and system architecture
- Implement data security and compliance best practices
- Conduct code reviews for quality assurance
- Integrate Hadoop with other data processing systems (e.g., HBase, Cassandra)
- Tune and optimize Hadoop ecosystem components
- Automate tasks using tools like Airflow for scheduling
- Keep updated with Hadoop ecosystem advancements and apply new technologies
- Provide training and support on Hadoop-related topics
- Participate in agile ceremonies and project planning activities
- Maintain and ensure the reliability of Hadoop infrastructure

The ideal candidate for the Hadoop Developer role is a highly skilled and experienced professional with a Bachelor's degree in Computer Science, Information Technology, or a related field and a proven track record in Hadoop development. They possess in-depth knowledge and hands-on experience with Hadoop ecosystem technologies such as HDFS, MapReduce, Yarn, Pig, Hive, HBase, Flume, and ZooKeeper, and are proficient in programming languages like Java, Python, or Scala. The candidate has a strong foundation in data warehousing, ETL processes, and data modeling, with practical experience using big data tools such as Apache Spark, Kafka, or Apache Nifi. They have a solid understanding of the Linux operating system, bash/shell scripting, and SQL. Additionally, they are well-versed in version control systems like Git and have a working knowledge of Agile development methodologies and tools like JIRA. The ideal candidate is not only technically proficient but also possesses strong problem-solving and analytical skills, exceptional attention to detail, and an innovative mindset. They are proactive, self-motivated, and able to adapt quickly to new technologies and methodologies. With excellent communication and interpersonal skills, the candidate is a collaborative team player capable of managing and prioritizing multiple tasks in high-pressure situations. They are passionate about big data and data analytics, committed to data security and privacy best practices, and dedicated to continuous learning and process optimization, making them an invaluable addition to the team. Familiarity with cloud platforms such as AWS, Google Cloud, or Azure and Hadoop-related certifications would be a plus.

- Write, maintain, and optimize Hadoop jobs
- Design, develop, and manage scalable data pipelines
- Perform data extraction, transformation, and loading (ETL) processes
- Monitor Hadoop cluster performance and troubleshoot issues
- Perform cluster resource management and high availability configurations
- Develop custom data processing and analysis applications using MapReduce, Hive, and Pig
- Collaborate with data scientists and analysts to implement data models and algorithms
- Conduct data validation and quality assurance checks
- Create and maintain technical documentation for data processes and systems
- Implement data security and privacy best practices
- Perform code reviews to ensure adherence to coding standards and best practices
- Integrate Hadoop-based applications with other data storage and processing frameworks (e.g., HBase, Cassandra)
- Tune job performance and optimize Hadoop ecosystem components
- Automate repetitive tasks and deploy airflow jobs for task scheduling
- Stay updated with the latest Hadoop ecosystem developments and incorporate new technologies when appropriate
- Provide support and training to other team members on Hadoop-related topics
- Participate in sprint planning, daily stand-ups, and retrospective meetings
- Manage and maintain Hadoop infrastructure, ensuring high availability and reliability

- Strong problem-solving and analytical skills
- Attention to detail and precision in work
- Proactive and self-motivated
- Ability to adapt to new technologies and methodologies quickly
- Collaborative team player with excellent interpersonal skills
- Strong communication skills, both written and verbal
- Demonstrated ability to manage and prioritize multiple tasks
- Capacity to work independently and deliver high-quality results
- Strong organizational and time management skills
- Continuous learner, eager to stay updated with emerging technologies
- Innovative mindset and ability to think critically
- Strong ethics and commitment to data security and privacy best practices
- Reliable and responsible, with a focus on consistency and quality
- Ability to handle high-pressure situations and meet tight deadlines
- Passion for big data and data analytics
- Commitment to continuous improvement and process optimization

- Competitive salary range ($X,000 - $Y,000 per year)
- Comprehensive health, dental, and vision insurance
- Generous paid time off and vacation days
- Flexible working hours and remote work options
- Retirement savings plan with company match
- Professional development opportunities, including courses and certifications
- Access to industry conferences and workshops
- Employee assistance programs for health and well-being
- Performance-based bonuses and incentives
- Stock options or equity (if applicable)
- Parental leave and family support benefits
- Life and disability insurance coverage
- Gym membership discounts or wellness programs
- Collaborative and inclusive work environment
- Opportunities for career growth and advancement
- Regular team-building activities and social events
- Employee recognition programs
- State-of-the-art technology and tools
- Support for continuing education and skill development
- Travel opportunities for project engagements and conferences (if applicable)