11+ Apache Drill Jobs in India

Data Platform engineering at Uber is looking for a strong Technical Lead (Level 5a Engineer) who has built high quality platforms and services that can operate at scale. 5a Engineer at Uber exhibits following qualities: 

• Demonstrate tech expertise › Demonstrate technical skills to go very deep or broad in solving classes of problems or creating broadly leverageable solutions. 
• Execute large scale projects › Define, plan and execute complex and impactful projects. You communicate the vision to peers and stakeholders.
• Collaborate across teams › Domain resource to engineers outside your team and help them leverage the right solutions. Facilitate technical discussions and drive to a consensus.
• Coach engineers › Coach and mentor less experienced engineers and deeply invest in their learning and success. You give and solicit feedback, both positive and negative, to others you work with to help improve the entire team.
• Tech leadership › Lead the effort to define the best practices in your immediate team, and help the broader organization establish better technical or business processes.

What You’ll Do

• Build a scalable, reliable, operable and performant data analytics platform for Uber’s engineers, data scientists, products and operations teams.
• Work alongside the pioneers of big data systems such as Hive, Yarn, Spark, Presto, Kafka, Flink to build out a highly reliable, performant, easy to use software system for Uber’s planet scale of data. 
• Become proficient of multi-tenancy, resource isolation, abuse prevention, self-serve debuggability aspects of a high performant, large scale, service while building these capabilities for Uber's engineers and operation folks.

What You’ll Need

• 7+ years experience in building large scale products, data platforms, distributed systems in a high caliber environment.
• Architecture: Identify and solve major architectural problems by going deep in your field or broad across different teams. Extend, improve, or, when needed, build solutions to address architectural gaps or technical debt.
• Software Engineering/Programming: Create frameworks and abstractions that are reliable and reusable. advanced knowledge of at least one programming language, and are happy to learn more. Our core languages are Java, Python, Go, and Scala.
• Data Engineering: Expertise in one of the big data analytics technologies we currently use such as Apache Hadoop (HDFS and YARN), Apache Hive, Impala, Drill, Spark, Tez, Presto, Calcite, Parquet, Arrow etc. Under the hood experience with similar systems such as Vertica, Apache Impala, Drill, Google Borg, Google BigQuery, Amazon EMR, Amazon RedShift, Docker, Kubernetes, Mesos etc.
• Execution & Results: You tackle large technical projects/problems that are not clearly defined. You anticipate roadblocks and have strategies to de-risk timelines. You orchestrate work that spans multiple teams and keep your stakeholders informed.
• A team player: You believe that you can achieve more on a team that the whole is greater than the sum of its parts. You rely on others’ candid feedback for continuous improvement.
• Business acumen: You understand requirements beyond the written word. Whether you’re working on an API used by other developers, an internal tool consumed by our operation teams, or a feature used by millions of customers, your attention to details leads to a delightful user experience.

Publicis Sapient Overview:

The Senior Associate People Senior Associate L1 in Data Engineering, you will translate client requirements into technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution 

.

Job Summary:

As Senior Associate L2 in Data Engineering, you will translate client requirements into technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution

The role requires a hands-on technologist who has strong programming background like Java / Scala / Python, should have experience in Data Ingestion, Integration and data Wrangling, Computation, Analytics pipelines and exposure to Hadoop ecosystem components. You are also required to have hands-on knowledge on at least one of AWS, GCP, Azure cloud platforms.

Role & Responsibilities:

Your role is focused on Design, Development and delivery of solutions involving:

• Data Integration, Processing & Governance

• Data Storage and Computation Frameworks, Performance Optimizations

• Analytics & Visualizations

• Infrastructure & Cloud Computing

• Data Management Platforms

• Implement scalable architectural models for data processing and storage

• Build functionality for data ingestion from multiple heterogeneous sources in batch & real-time mode

• Build functionality for data analytics, search and aggregation

Experience Guidelines:

Mandatory Experience and Competencies:

# Competency

1.Overall 5+ years of IT experience with 3+ years in Data related technologies

2.Minimum 2.5 years of experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP)

3.Hands-on experience with the Hadoop stack – HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data pipeline.

4.Strong experience in at least of the programming language Java, Scala, Python. Java preferable

5.Hands-on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDb, Cassandra, AWS Redshift, Azure SQLDW, GCP BigQuery etc

6.Well-versed and working knowledge with data platform related services on at least 1 cloud platform, IAM and data security

Preferred Experience and Knowledge (Good to Have):

# Competency

1.Good knowledge of traditional ETL tools (Informatica, Talend, etc) and database technologies (Oracle, MySQL, SQL Server, Postgres) with hands on experience

2.Knowledge on data governance processes (security, lineage, catalog) and tools like Collibra, Alation etc

3.Knowledge on distributed messaging frameworks like ActiveMQ / RabbiMQ / Solace, search & indexing and Micro services architectures

4.Performance tuning and optimization of data pipelines

5.CI/CD – Infra provisioning on cloud, auto build & deployment pipelines, code quality

6.Cloud data specialty and other related Big data technology certifications

Personal Attributes:

• Strong written and verbal communication skills

• Articulation skills

• Good team player

• Self-starter who requires minimal oversight

• Ability to prioritize and manage multiple tasks

• Process orientation and the ability to define and set up processes

The role is with a Fintech Credit Card company based in Pune within the Decision Science team. (OneCard )

About

Credit cards haven't changed much for over half a century so our team of seasoned bankers, technologists, and designers set out to redefine the credit card for you - the consumer. The result is OneCard - a credit card reimagined for the mobile generation. OneCard is India's best metal credit card built with full-stack tech. It is backed by the principles of simplicity, transparency, and giving back control to the user.

The Engineering Challenge

“Re-imaging credit and payments from First Principles”

Payments is an interesting engineering challenge in itself with requirements of low latency, transactional guarantees, security, and high scalability. When we add credit and engagement into the mix, the challenge becomes even more interesting with underwriting and recommendation algorithms working on large data sets. We have eliminated the current call center, sales agent, and SMS-based processes with a mobile app that puts the customers in complete control. To stay agile, the entire stack is built on the cloud with modern technologies.

Purpose of Role :

- Develop and implement the collection analytics and strategy function for the credit cards. Use analysis and customer insights to develop optimum strategy.

CANDIDATE PROFILE :

- Successful candidates will have in-depth knowledge of statistical modelling/data analysis tools (Python, R etc.), techniques. They will be an adept communicator with good interpersonal skills to work with senior stake holders in India to grow revenue primarily through identifying / delivering / creating new, profitable analytics solutions.

We are looking for someone who:

- Proven track record in collection and risk analytics preferably in Indian BFSI industry. This is a must.

- Identify & deliver appropriate analytics solutions

- Experienced in Analytics team management

Essential Duties and Responsibilities :

- Responsible for delivering high quality analytical and value added services

- Responsible for automating insights and proactive actions on them to mitigate collection Risk.

- Work closely with the internal team members to deliver the solution

- Engage Business/Technical Consultants and delivery teams appropriately so that there is a shared understanding and agreement as to deliver proposed solution

- Use analysis and customer insights to develop value propositions for customers

- Maintain and enhance the suite of suitable analytics products.

- Actively seek to share knowledge within the team

- Share findings with peers from other teams and management where required

- Actively contribute to setting best practice processes.

Knowledge, Experience and Qualifications :

Knowledge :

- Good understanding of collection analytics preferably in Retail lending industry.

- Knowledge of statistical modelling/data analysis tools (Python, R etc.), techniques and market trends

- Knowledge of different modelling frameworks like Linear Regression, Logistic Regression, Multiple Regression, LOGIT, PROBIT, time- series modelling, CHAID, CART etc.

- Knowledge of Machine learning & AI algorithms such as Gradient Boost, KNN, etc.

- Understanding of decisioning and portfolio management in banking and financial services would be added advantage

- Understanding of credit bureau would be an added advantage

Experience :

- 4 to 8 years of work experience in core analytics function of a large bank / consulting firm.

- Experience on working on Collection analytics is must

- Experience on handling large data volumes using data analysis tools and generating good data insights

- Demonstrated ability to communicate ideas and analysis results effectively both verbally and in writing to technical and non-technical audiences

- Excellent communication, presentation and writing skills Strong interpersonal skills

- Motivated to meet and exceed stretch targets

- Ability to make the right judgments in the face of complexity and uncertainty

- Excellent relationship and networking skills across our different business and geographies

Qualifications :

- Masters degree in Statistics, Mathematics, Economics, Business Management or Engineering from a reputed college

• 4-10 years of experience in software development.
• At least 2 years of relevant work experience on large scale Data applications.
• Strong coding experience in Java is mandatory
• Good aptitude, strong problem solving abilities, and analytical skills, ability to take ownership as appropriate
• Should be able to do coding, debugging, performance tuning and deploying the apps to Prod.
• Should have good working experience on
• o Hadoop ecosystem (HDFS, Hive, Yarn, File formats like Avro/Parquet)
• o Kafka
• o J2EE Frameworks (Spring/Hibernate/REST)
• o Spark Streaming or any other streaming technology.
• Strong coding experience in Java is mandatory
• Ability to work on the sprint stories to completion along with Unit test case coverage.
• Experience working in Agile Methodology
• Excellent communication and coordination skills
• Knowledgeable (and preferred hands on) - UNIX environments, different continuous integration tools.
• Must be able to integrate quickly into the team and work independently towards team goals

• Take the complete responsibility of the sprint stories' execution
• Be accountable for the delivery of the tasks in the defined timelines with good quality.
• Follow the processes for project execution and delivery.
• Follow agile methodology
• Work with the team lead closely and contribute to the smooth delivery of the project.
• Understand/define the architecture and discuss the pros-cons of the same with the team
• Involve in the brainstorming sessions and suggest improvements in the architecture/design.
• Work with other team leads to get the architecture/design reviewed.
• Work with the clients and counter-parts (in US) of the project.
• Keep all the stakeholders updated about the project/task status/risks/issues if there are any.

About LodgIQ

LodgIQ is led by a team of experienced hospitality technology experts, data scientists and product domain experts. Seed funded by Highgate Ventures, a venture capital platform focused on early stage technology investments in the hospitality industry and Trilantic Capital Partners, a global private equity firm, LodgIQ has made a significant investment in advanced machine learning platforms and data science.

Title : Data Scientist

• Apply Data Science and Machine Learning to a REAL-LIFE  problem  - “Predict Guest Arrivals and Determine Best Prices for Hotels”
• Apply advanced analytics in a BIG Data Environment – AWS, MongoDB, SKLearn
• Help scale up the product in a global offering across 100+  global markets

• Minimum 3 years of experience with advanced data analytic techniques, including data mining, machine learning, statistical analysis, and optimization.    Student projects are acceptable.
• At least 1 year of experience with Python / Numpy / Pandas / Scipy/ MatPlotLib / Scikit-Learn
•   Experience in working with massive data sets, including structured and unstructured with at least 1 prior engagement involving data gathering, data cleaning, data mining, and data visualization
•   Solid grasp over  optimization techniques
•   Master's  or PhD degree in Business Analytics.  Data science, Statistics or Mathematics
•   Ability to show a track record of solving large, complex problems

   3+ years of experience in deployment, monitoring, tuning, and administration of high concurrency MySQL production databases.

• Solid understanding of writing optimized SQL queries on MySQL databases
• Understanding of AWS, VPC, networking, security groups, IAM, and roles.
• Expertise in scripting in Python or Shell/Powershell
• Must have experience in large scale data migrations
• Excellent communication skills.

Intro

Our data and risk team is the core pillar of our business that harnesses alternative data sources to guide the decisions we make at Rely. The team designs, architects, as well as develop and maintain a scalable data platform the powers our machine learning models. Be part of a team that will help millions of consumers across Asia, to be effortlessly in control of their spending and make better decisions.

What will you do
The data engineer is focused on making data correct and accessible, and building scalable systems to access/process it. Another major responsibility is helping AI/ML Engineers write better code.

• Optimize and automate ingestion processes for a variety of data sources such as: click stream, transactional and many other sources.

• Create and maintain optimal data pipeline architecture and ETL processes
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Develop data pipeline and infrastructure to support real-time decisions
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data' technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.
  
  

What will you need
• 2+ hands-on experience building and implementation of large scale production pipeline and Data Warehouse
• Experience dealing with large scale

• Proficiency in writing and debugging complex SQLs
• Experience working with AWS big data tools
  • Ability to lead the project and implement best data practises and technology

Data Pipelining

• Strong command in building & optimizing data pipelines, architectures and data sets
• Strong command on relational SQL & noSQL databases including Postgres
• Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

Big Data: Strong experience in big data tools & applications

• Tools: Hadoop, Spark, HDFS etc
• AWS cloud services: EC2, EMR, RDS, Redshift
• Stream-processing systems: Storm, Spark-Streaming, Flink etc.
• Message queuing: RabbitMQ, Spark etc
  
  

Software Development & Debugging

• Strong experience in object-oriented programming/object function scripting languages: Python, Java, C++, Scala, etc
• Strong hold on data structures & algorithms

What would be a bonus

• Prior experience working in a fast-growth Startup
• Prior experience in the payments, fraud, lending, advertising companies dealing with large scale data