11+ Amazon EMR Jobs in Delhi, NCR and Gurgaon | Amazon EMR Job openings in Delhi, NCR and Gurgaon

• Mandatory - Hands on experience in Python and PySpark.

• Build pySpark applications using Spark Dataframes in Python using Jupyter notebook and PyCharm(IDE).

• Worked on optimizing spark jobs that processes huge volumes of data.

• Hands on experience in version control tools like Git.

• Worked on Amazon’s Analytics services like Amazon EMR, Lambda function etc

• Worked on Amazon’s Compute services like Amazon Lambda, Amazon EC2 and Amazon’s Storage service like S3 and few other services like SNS.

• Experience/knowledge of bash/shell scripting will be a plus.

• Experience in working with fixed width, delimited , multi record file formats etc.

• Hands on experience in tools like Jenkins to build, test and deploy the applications

• Awareness of Devops concepts and be able to work in an automated release pipeline environment.

• Excellent debugging skills.

Publicis Sapient Overview:

The Senior Associate People Senior Associate L1 in Data Engineering, you will translate client requirements into technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution 

.

Job Summary:

As Senior Associate L2 in Data Engineering, you will translate client requirements into technical design, and implement components for data engineering solution. Utilize deep understanding of data integration and big data design principles in creating custom solutions or implementing package solutions. You will independently drive design discussions to insure the necessary health of the overall solution

The role requires a hands-on technologist who has strong programming background like Java / Scala / Python, should have experience in Data Ingestion, Integration and data Wrangling, Computation, Analytics pipelines and exposure to Hadoop ecosystem components. You are also required to have hands-on knowledge on at least one of AWS, GCP, Azure cloud platforms.

Role & Responsibilities:

Your role is focused on Design, Development and delivery of solutions involving:

• Data Integration, Processing & Governance

• Data Storage and Computation Frameworks, Performance Optimizations

• Analytics & Visualizations

• Infrastructure & Cloud Computing

• Data Management Platforms

• Implement scalable architectural models for data processing and storage

• Build functionality for data ingestion from multiple heterogeneous sources in batch & real-time mode

• Build functionality for data analytics, search and aggregation

Experience Guidelines:

Mandatory Experience and Competencies:

# Competency

1.Overall 5+ years of IT experience with 3+ years in Data related technologies

2.Minimum 2.5 years of experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP)

3.Hands-on experience with the Hadoop stack – HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data pipeline.

4.Strong experience in at least of the programming language Java, Scala, Python. Java preferable

5.Hands-on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDb, Cassandra, AWS Redshift, Azure SQLDW, GCP BigQuery etc

6.Well-versed and working knowledge with data platform related services on at least 1 cloud platform, IAM and data security

Preferred Experience and Knowledge (Good to Have):

# Competency

1.Good knowledge of traditional ETL tools (Informatica, Talend, etc) and database technologies (Oracle, MySQL, SQL Server, Postgres) with hands on experience

2.Knowledge on data governance processes (security, lineage, catalog) and tools like Collibra, Alation etc

3.Knowledge on distributed messaging frameworks like ActiveMQ / RabbiMQ / Solace, search & indexing and Micro services architectures

4.Performance tuning and optimization of data pipelines

5.CI/CD – Infra provisioning on cloud, auto build & deployment pipelines, code quality

6.Cloud data specialty and other related Big data technology certifications

Personal Attributes:

• Strong written and verbal communication skills

• Articulation skills

• Good team player

• Self-starter who requires minimal oversight

• Ability to prioritize and manage multiple tasks

• Process orientation and the ability to define and set up processes

1. Data Engineer

 Required skill set: AWS GLUE, AWS LAMBDA, AWS SNS/SQS, AWS ATHENA, SPARK, SNOWFLAKE, PYTHON

Mandatory Requirements  

• Experience in AWS Glue
• Experience in Apache Parquet 
• Proficient in AWS S3 and data lake 
• Knowledge of Snowflake
• Understanding of file-based ingestion best practices.
• Scripting language - Python & pyspark 

CORE RESPONSIBILITIES 

• Create and manage cloud resources in AWS 
• Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies 
• Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform 
• Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations 
• Develop an infrastructure to collect, transform, combine and publish/distribute customer data.
• Define process improvement opportunities to optimize data collection, insights and displays.
• Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible 
• Identify and interpret trends and patterns from complex data sets 
• Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders. 
• Key participant in regular Scrum ceremonies with the agile teams  
• Proficient at developing queries, writing reports and presenting findings 
• Mentor junior members and bring best industry practices 

 QUALIFICATIONS 

• 5-7+ years’ experience as data engineer in consumer finance or equivalent industry (consumer loans, collections, servicing, optional product, and insurance sales) 
• Strong background in math, statistics, computer science, data science or related discipline
• Advanced knowledge one of language: Java, Scala, Python, C# 
• Production experience with: HDFS, YARN, Hive, Spark, Kafka, Oozie / Airflow, Amazon Web Services (AWS), Docker / Kubernetes, Snowflake  
• Proficient with
• Data mining/programming tools (e.g. SAS, SQL, R, Python)
• Database technologies (e.g. PostgreSQL, Redshift, Snowflake. and Greenplum)
• Data visualization (e.g. Tableau, Looker, MicroStrategy)
• Comfortable learning about and deploying new technologies and tools. 
• Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines. 
• Good written and oral communication skills and ability to present results to non-technical audiences 
• Knowledge of business intelligence and analytical tools, technologies and techniques.

Familiarity and experience in the following is a plus:  

• AWS certification
• Spark Streaming 
• Kafka Streaming / Kafka Connect 
• ELK Stack 
• Cassandra / MongoDB 
• CI/CD: Jenkins, GitLab, Jira, Confluence other related tools

Who Are We

A research-oriented company with expertise in computer vision and artificial intelligence, at its core, Orbo is a comprehensive platform of AI-based visual enhancement stack. This way, companies can find a suitable product as per their need where deep learning powered technology can automatically improve their Imagery.

ORBO's solutions are helping BFSI, beauty and personal care digital transformation and Ecommerce image retouching industries in multiple ways.

WHY US

• Join top AI company
• Grow with your best companions
• Continuous pursuit of excellence, equality, respect
• Competitive compensation and benefits

You'll be a part of the core team and will be working directly with the founders in building and iterating upon the core products that make cameras intelligent and images more informative.

To learn more about how we work, please check out

https://www.orbo.ai/.

Description:

We are looking for a computer vision engineer to lead our team in developing a factory floor analytics SaaS product. This would be a fast-paced role and the person will get an opportunity to develop an industrial grade solution from concept to deployment.

Responsibilities:

• Research and develop computer vision solutions for industries (BFSI, Beauty and personal care, E-commerce, Defence etc.)
• Lead a team of ML engineers in developing an industrial AI product from scratch
• Setup end-end Deep Learning pipeline for data ingestion, preparation, model training, validation and deployment
• Tune the models to achieve high accuracy rates and minimum latency
• Deploying developed computer vision models on edge devices after optimization to meet customer requirements

Requirements:

• Bachelor’s degree
• Understanding about depth and breadth of computer vision and deep learning algorithms.
• 4+ years of industrial experience in computer vision and/or deep learning
• Experience in taking an AI product from scratch to commercial deployment.
• Experience in Image enhancement, object detection, image segmentation, image classification algorithms
• Experience in deployment with OpenVINO, ONNXruntime and TensorRT
• Experience in deploying computer vision solutions on edge devices such as Intel Movidius and Nvidia Jetson
• Experience with any machine/deep learning frameworks like Tensorflow, and PyTorch.
• Proficient understanding of code versioning tools, such as Git

Our perfect candidate is someone that:

• is proactive and an independent problem solver
• is a constant learner. We are a fast growing start-up. We want you to grow with us!
• is a team player and good communicator

What We Offer:

• You will have fun working with a fast-paced team on a product that can impact the business model of E-commerce and BFSI industries. As the team is small, you will easily be able to see a direct impact of what you build on our customers (Trust us - it is extremely fulfilling!)
• You will be in charge of what you build and be an integral part of the product development process
• Technical and financial growth!

1. Sr. Data Engineer:

 Core Skills – Data Engineering, Big Data, Pyspark, Spark SQL and Python

Candidate with prior Palantir Cloud Foundry OR Clinical Trial Data Model background is preferred

Major accountabilities:

• Responsible for Data Engineering, Foundry Data Pipeline Creation, Foundry Analysis & Reporting, Slate Application development, re-usable code development & management and Integrating Internal or External System with Foundry for data ingestion with high quality.
• Have good understanding on Foundry Platform landscape and it’s capabilities
• Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.
• Defines company data assets (data models), Pyspark, spark SQL, jobs to populate data models.
• Designs data integrations and data quality framework.
• Design & Implement integration with Internal, External Systems, F1 AWS platform using Foundry Data Connector or Magritte Agent
• Collaboration with data scientists, data analyst and technology teams to document and leverage their understanding of the Foundry integration with different data sources - Actively participate in agile work practices
• Coordinating with Quality Engineer to ensure the all quality controls, naming convention & best practices have been followed

Desired Candidate Profile :

• Strong data engineering background
• Experience with Clinical Data Model is preferred
• Experience in
• SQL Server ,Postgres, Cassandra, Hadoop, and Spark for distributed data storage and parallel computing
• Java and Groovy for our back-end applications and data integration tools
• Python for data processing and analysis
• Cloud infrastructure based on AWS EC2 and S3
• 7+ years IT experience, 2+ years’ experience in Palantir Foundry Platform, 4+ years’ experience in Big Data platform
• 5+ years of Python and Pyspark development experience
• Strong troubleshooting and problem solving skills
• BTech or master's degree in computer science or a related technical field
• Experience designing, building, and maintaining big data pipelines systems
• Hands-on experience on Palantir Foundry Platform and Foundry custom Apps development
• Able to design and implement data integration between Palantir Foundry and external Apps based on Foundry data connector framework
• Hands-on in programming languages primarily Python, R, Java, Unix shell scripts
• Hand-on experience in AWS / Azure cloud platform and stack
• Strong in API based architecture and concept, able to do quick PoC using API integration and development
• Knowledge of machine learning and AI
• Skill and comfort working in a rapidly changing environment with dynamic objectives and iteration with users.

 Demonstrated ability to continuously learn, work independently, and make decisions with minimal supervision

Responsibilities:

• Exploring and visualizing data to gain an understanding of it, then identifying differences in data distribution that could affect performance when deploying the model in the real world.
• Verifying data quality, and/or ensuring it via data cleaning.
• Able to adapt and work fast in producing the output which upgrades the decision making of stakeholders using ML.
• To design and develop Machine Learning systems and schemes. 
• To perform statistical analysis and fine-tune models using test results.
• To train and retrain ML systems and models as and when necessary. 
• To deploy ML models in production and maintain the cost of cloud infrastructure.
• To develop Machine Learning apps according to client and data scientist requirements.
• To analyze the problem-solving capabilities and use-cases of ML algorithms and rank them by how successful they are in meeting the objective.

Technical Knowledge:

• Worked with real time problems, solved them using ML and deep learning models deployed in real time and should have some awesome projects under his belt to showcase. 
• Proficiency in Python and experience with working with Jupyter Framework, Google collab and cloud hosted notebooks such as AWS sagemaker, DataBricks etc.
• Proficiency in working with libraries Sklearn, Tensorflow, Open CV2, Pyspark,  Pandas, Numpy and related libraries.
• Expert in visualising and manipulating complex datasets.
• Proficiency in working with visualisation libraries such as seaborn, plotly, matplotlib etc.
• Proficiency in Linear Algebra, statistics and probability required for Machine Learning.
• Proficiency in ML Based algorithms for example, Gradient boosting, stacked Machine learning, classification algorithms and deep learning algorithms. Need to have experience in hypertuning various models and comparing the results of algorithm performance.
• Big data Technologies such as Hadoop stack and Spark. 
• Basic use of clouds (VM’s example EC2).
• Brownie points for Kubernetes and Task Queues.      
• Strong written and verbal communications.
• Experience working in an Agile environment.