11+ HiveQL Jobs in Chennai | HiveQL Job openings in Chennai

Role Summary/Purpose:

We are looking for a Developer/Senior Developers to be a part of building advanced analytical platform leveraging Big Data technologies and transform the legacy systems. This role is an exciting, fast-paced, constantly changing and challenging work environment, and will play an important role in resolving and influencing high-level decisions.

Requirements:

• The candidate must be a self-starter, who can work under general guidelines in a fast-spaced environment.
• Overall minimum of 4 to 8 year of software development experience and 2 years in Data Warehousing domain knowledge
• Must have 3 years of hands-on working knowledge on Big Data technologies such as Hadoop, Hive, Hbase, Spark, Kafka, Spark Streaming, SCALA etc…
• Excellent knowledge in SQL & Linux Shell scripting
• Bachelors/Master’s/Engineering Degree from a well-reputed university.
• Strong communication, Interpersonal, Learning and organizing skills matched with the ability to manage stress, Time, and People effectively
• Proven experience in co-ordination of many dependencies and multiple demanding stakeholders in a complex, large-scale deployment environment
• Ability to manage a diverse and challenging stakeholder community
• Diverse knowledge and experience of working on Agile Deliveries and Scrum teams.

Responsibilities

• Should works as a senior developer/individual contributor based on situations
• Should be part of SCRUM discussions and to take requirements
• Adhere to SCRUM timeline and deliver accordingly
• Participate in a team environment for the design, development and implementation
• Should take L3 activities on need basis
• Prepare Unit/SIT/UAT testcase and log the results
• Co-ordinate SIT and UAT Testing. Take feedbacks and provide necessary remediation/recommendation in time.
• Quality delivery and automation should be a top priority
• Co-ordinate change and deployment in time
• Should create healthy harmony within the team
• Owns interaction points with members of core team (e.g.BA team, Testing and business team) and any other relevant stakeholders

Job Responsibilities

• Design, build & test ETL processes using Python & SQL for the corporate data warehouse
• Inform, influence, support, and execute our product decisions
• Maintain advertising data integrity by working closely with R&D to organize and store data in a format that provides accurate data and allows the business to quickly identify issues.
• Evaluate and prototype new technologies in the area of data processing
• Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams
• High energy level, strong team player and good work ethic
• Data analysis, understanding of business requirements and translation into logical pipelines & processes
• Identification, analysis & resolution of production & development bugs
• Support the release process including completing & reviewing documentation
• Configure data mappings & transformations to orchestrate data integration & validation
• Provide subject matter expertise
• Document solutions, tools & processes
• Create & support test plans with hands-on testing
• Peer reviews of work developed by other data engineers within the team
• Establish good working relationships & communication channels with relevant departments

Skills and Qualifications we look for

• University degree 2.1 or higher (or equivalent) in a relevant subject. Master’s degree in any data subject will be a strong advantage.
• 4 - 6 years experience with data engineering.
• Strong coding ability and software development experience in Python.
• Strong hands-on experience with SQL and Data Processing.
• Google cloud platform (Cloud composer, Dataflow, Cloud function, Bigquery, Cloud storage, dataproc)
• Good working experience in any one of the ETL tools (Airflow would be preferable).
• Should possess strong analytical and problem solving skills.
• Good to have skills - Apache pyspark, CircleCI, Terraform
• Motivated, self-directed, able to work with ambiguity and interested in emerging technologies, agile and collaborative processes.
• Understanding & experience of agile / scrum delivery methodology

AWS Glue Developer 

Work Experience: 6 to 8 Years

Work Location:  Noida, Bangalore, Chennai & Hyderabad

Must Have Skills: AWS Glue, DMS, SQL, Python, PySpark, Data integrations and Data Ops, 

Job Reference ID:BT/F21/IND

Job Description:

Design, build and configure applications to meet business process and application requirements.

Responsibilities:

7 years of work experience with ETL, Data Modelling, and Data Architecture Proficient in ETL optimization, designing, coding, and tuning big data processes using Pyspark Extensive experience to build data platforms on AWS using core AWS services Step function, EMR, Lambda, Glue and Athena, Redshift, Postgres, RDS etc and design/develop data engineering solutions. Orchestrate using Airflow.

Technical Experience:

Hands-on experience on developing Data platform and its components Data Lake, cloud Datawarehouse, APIs, Batch and streaming data pipeline Experience with building data pipelines and applications to stream and process large datasets at low latencies.

➢ Enhancements, new development, defect resolution and production support of Big data ETL development using AWS native services.

➢ Create data pipeline architecture by designing and implementing data ingestion solutions.

➢ Integrate data sets using AWS services such as Glue, Lambda functions/ Airflow.

➢ Design and optimize data models on AWS Cloud using AWS data stores such as Redshift, RDS, S3, Athena.

➢ Author ETL processes using Python, Pyspark.

➢ Build Redshift Spectrum direct transformations and data modelling using data in S3.

➢ ETL process monitoring using CloudWatch events.

➢ You will be working in collaboration with other teams. Good communication must.

➢ Must have experience in using AWS services API, AWS CLI and SDK

Professional Attributes:

➢ Experience operating very large data warehouses or data lakes Expert-level skills in writing and optimizing SQL Extensive, real-world experience designing technology components for enterprise solutions and defining solution architectures and reference architectures with a focus on cloud technology.

➢ Must have 6+ years of big data ETL experience using Python, S3, Lambda, Dynamo DB, Athena, Glue in AWS environment.

➢ Expertise in S3, RDS, Redshift, Kinesis, EC2 clusters highly desired.

Qualification:

➢ Degree in Computer Science, Computer Engineering or equivalent.

Salary: Commensurate with experience and demonstrated competence

Job Location: Hyderabad/Bangalore/ Chennai/Pune/Nagpur

Notice period: Immediate - 15 days

1.      Python Developer with Snowflake

Job Description :

1. 5.5+ years of Strong Python Development Experience with Snowflake.
2. Strong hands of experience with SQL ability to write complex queries.
3. Strong understanding of how to connect to Snowflake using Python, should be able to handle any type of files
4.  Development of Data Analysis, Data Processing engines using Python
5. Good Experience in Data Transformation using Python. 
6.  Experience in Snowflake data load using Python.
7.  Experience in creating user-defined functions in Snowflake.
8.  Snowsql implementation.
9.  Knowledge of query performance tuning will be added advantage.
10. Good understanding of Datawarehouse (DWH) concepts.
11.  Interpret/analyze business requirements & functional specification
12.  Good to have DBT, FiveTran, and AWS Knowledge.

THE IDEAL CANDIDATE WILL

• Engage with executive level stakeholders from client's team to translate business problems to high level solution approach
• Partner closely with practice, and technical teams to craft well-structured comprehensive proposals/ RFP responses clearly highlighting Tredence’s competitive strengths relevant to Client's selection criteria
• Actively explore the client’s business and formulate solution ideas that can improve process efficiency and cut cost, or achieve growth/revenue/profitability targets faster
• Work hands-on across various MLOps problems and provide thought leadership
• Grow and manage large teams with diverse skillsets
• Collaborate, coach, and learn with a growing team of experienced Machine Learning Engineers and Data Scientists

ELIGIBILITY CRITERIA

• BE/BTech/MTech (Specialization/courses in ML/DS)
• At-least 7+ years of Consulting services delivery experience
• Very strong problem-solving skills & work ethics
• Possesses strong analytical/logical thinking, storyboarding and executive communication skills
• 5+ years of experience in Python/R, SQL
• 5+ years of experience in NLP algorithms, Regression & Classification Modelling, Time Series Forecasting
• Hands on work experience in DevOps
• Should have good knowledge in different deployment type like PaaS, SaaS, IaaS
• Exposure on cloud technologies like Azure, AWS or GCP
• Knowledge in python and packages for data analysis (scikit-learn, scipy, numpy, pandas, matplotlib).
• Knowledge of Deep Learning frameworks: Keras, Tensorflow, PyTorch, etc
• Experience with one or more Container-ecosystem (Docker, Kubernetes)
• Experience in building orchestration pipeline to convert plain python models into a deployable API/RESTful endpoint.
• Good understanding of OOP & Data Structures concepts

Nice to Have:

• Exposure to deployment strategies like: Blue/Green, Canary, AB Testing, Multi-arm Bandit
• Experience in Helm is a plus
• Strong understanding of data infrastructure, data warehouse, or data engineering

You can expect to –

• Work with world’ biggest retailers and help them solve some of their most critical problems. Tredence is a preferred analytics vendor for some of the largest Retailers across the globe
• Create multi-million Dollar business opportunities by leveraging impact mindset, cutting edge solutions and industry best practices.
• Work in a diverse environment that keeps evolving
• Hone your entrepreneurial skills as you contribute to growth of the organization

Function : Sr. DB Developer

Location : India/Gurgaon/Tamilnadu

>> THE INDIVIDUAL

• Have a strong background in data platform creation and management.
• Possess in-depth knowledge of Data Management, Data Modelling, Ingestion  - Able to develop data models and ingestion frameworks based on client requirements and advise on system optimization.
• Hands-on experience in SQL database (PostgreSQL) and No-SQL database (MongoDB) 
• Hands-on experience in performance tuning of DB
• Good to have knowledge of database setup in cluster node
• Should be well versed with data security aspects and data governance framework
• Hands-on experience in Spark, Airflow, ELK.
• Good to have knowledge on any data cleansing tool like apache Griffin
• Preferably getting involved during project implementation so have a background on business knowledge and technical requirement as well.
• Strong analytical and problem-solving skills. Have exposure to data analytics skills and knowledge of advanced data analytical tools will be an advantage.
• Strong written and verbal communication skills (presentation skills).
• Certifications in the above technologies is preferred.

>> Qualification

1. Tech /B.E. / MCA /M. Tech from a reputed institute.

Experience of Data Management, Data Modelling, Ingestion for more than 4 years. Total experience of 8-10 Years

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools: Experience with big
  • data tools: Hadoop, Spark, Kafka, etc.
  • Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
  • Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
  • Experience with AWS cloud services: EC2, EMR, RDS, Redshift
  • Experience with stream-processing systems: Storm, Spark-Streaming, etc.
  • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

Position: Big Data Engineer

What You'll Do

Punchh is seeking to hire Big Data Engineer at either a senior or tech lead level. Reporting to the Director of Big Data, he/she will play a critical role in leading Punchh’s big data innovations. By leveraging prior industrial experience in big data, he/she will help create cutting-edge data and analytics products for Punchh’s business partners.

This role requires close collaborations with data, engineering, and product organizations. His/her job functions include

• Work with large data sets and implement sophisticated data pipelines with both structured and structured data.
• Collaborate with stakeholders to design scalable solutions.
• Manage and optimize our internal data pipeline that supports marketing, customer success and data science to name a few.
• A technical leader of Punchh’s big data platform that supports AI and BI products.
• Work with infra and operations team to monitor and optimize existing infrastructure 
• Occasional business travels are required.

What You'll Need

• 5+ years of experience as a Big Data engineering professional, developing scalable big data solutions.
• Advanced degree in computer science, engineering or other related fields.
• Demonstrated strength in data modeling, data warehousing and SQL.
• Extensive knowledge with cloud technologies, e.g. AWS and Azure.
• Excellent software engineering background. High familiarity with software development life cycle. Familiarity with GitHub/Airflow.
• Advanced knowledge of big data technologies, such as programming language (Python, Java), relational (Postgres, mysql), NoSQL (Mongodb), Hadoop (EMR) and streaming (Kafka, Spark).
• Strong problem solving skills with demonstrated rigor in building and maintaining a complex data pipeline.
• Exceptional communication skills and ability to articulate a complex concept with thoughtful, actionable recommendations.

• Should have good hands-on experience in Informatica MDM Customer 360, Data Integration(ETL) using PowerCenter, Data Quality.
• Must have strong skills in Data Analysis, Data Mapping for ETL processes, and Data Modeling.
• Experience with the SIF framework including real-time integration
• Should have experience in building C360 Insights using Informatica
• Should have good experience in creating performant design using Mapplets, Mappings, Workflows for Data Quality(cleansing), ETL.
• Should have experience in building different data warehouse architecture like Enterprise,
• Federated, and Multi-Tier architecture.
• Should have experience in configuring Informatica Data Director in reference to the Data
• Governance of users, IT Managers, and Data Stewards.
• Should have good knowledge in developing complex PL/SQL queries.
• Should have working experience on UNIX and shell scripting to run the Informatica workflows and to control the ETL flow.
• Should know about Informatica Server installation and knowledge on the Administration console.
• Working experience with Developer with Administration is added knowledge.
• Working experience in Amazon Web Services (AWS) is an added advantage. Particularly on AWS S3, Data pipeline, Lambda, Kinesis, DynamoDB, and EMR.
• Should be responsible for the creation of automated BI solutions, including requirements, design,development, testing, and deployment