11+ Apache HBase Jobs in Chennai | Apache HBase Job openings in Chennai

Location:  Chennai
Education: BE/BTech
Experience: Minimum 3+ years of experience as a Data Scientist/Data Engineer

Domain knowledge: Data cleaning, modelling, analytics, statistics, machine learning, AI

Requirements:

• To be part of Digital Manufacturing and Industrie 4.0 projects across client group of companies
• Design and develop AI//ML models to be deployed across factories
• Knowledge on Hadoop, Apache Spark, MapReduce, Scala, Python programming, SQL and NoSQL databases is required
• Should be strong in statistics, data analysis, data modelling, machine learning techniques and Neural Networks
• Prior experience in developing AI and ML models is required
• Experience with data from the Manufacturing Industry would be a plus

Roles and Responsibilities:

• Develop AI and ML models for the Manufacturing Industry with a focus on Energy, Asset Performance Optimization and Logistics
• Multitasking, good communication necessary
• Entrepreneurial attitude

• Job Location:                      Chennai
• Reporting to:                      Team Leader, Energy Management System

Title: Platform Engineer Location: Chennai Work Mode: Hybrid (Remote and Chennai Office) Experience: 4+ years Budget: 16 - 18 LPA

Responsibilities:

• Parse data using Python, create dashboards in Tableau.
• Utilize Jenkins for Airflow pipeline creation and CI/CD maintenance.
• Migrate Datastage jobs to Snowflake, optimize performance.
• Work with HDFS, Hive, Kafka, and basic Spark.
• Develop Python scripts for data parsing, quality checks, and visualization.
• Conduct unit testing and web application testing.
• Implement Apache Airflow and handle production migration.
• Apply data warehousing techniques for data cleansing and dimension modeling.

Requirements:

• 4+ years of experience as a Platform Engineer.
• Strong Python skills, knowledge of Tableau.
• Experience with Jenkins, Snowflake, HDFS, Hive, and Kafka.
• Proficient in Unix Shell Scripting and SQL.
• Familiarity with ETL tools like DataStage and DMExpress.
• Understanding of Apache Airflow.
• Strong problem-solving and communication skills.

Note: Only candidates willing to work in Chennai and available for immediate joining will be considered. Budget for this position is 16 - 18 LPA.

Data Engineer- Senior

Cubera is a data company revolutionizing big data analytics and Adtech through data share value principles wherein the users entrust their data to us. We refine the art of understanding, processing, extracting, and evaluating the data that is entrusted to us. We are a gateway for brands to increase their lead efficiency as the world moves towards web3.

What are you going to do?

Design & Develop high performance and scalable solutions that meet the needs of our customers.

Closely work with the Product Management, Architects and cross functional teams.

Build and deploy large-scale systems in Java/Python.

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Create data tools for analytics and data scientist team members that assist them in building and optimizing their algorithms.

Follow best practices that can be adopted in Bigdata stack.

Use your engineering experience and technical skills to drive the features and mentor the engineers.

What are we looking for ( Competencies) :

Bachelor’s degree in computer science, computer engineering, or related technical discipline.

Overall 5 to 8 years of programming experience in Java, Python including object-oriented design.

Data handling frameworks: Should have a working knowledge of one or more data handling frameworks like- Hive, Spark, Storm, Flink, Beam, Airflow, Nifi etc.

Data Infrastructure: Should have experience in building, deploying and maintaining applications on popular cloud infrastructure like AWS, GCP etc.

Data Store: Must have expertise in one of general-purpose No-SQL data stores like Elasticsearch, MongoDB, Redis, RedShift, etc.

Strong sense of ownership, focus on quality, responsiveness, efficiency, and innovation.

Ability to work with distributed teams in a collaborative and productive manner.

Benefits:

Competitive Salary Packages and benefits.

Collaborative, lively and an upbeat work environment with young professionals.

Job Category: Development

Job Type: Full Time

Job Location: Bangalore

Responsibilities:

• Must be able to write quality code and build secure, highly available systems.
• Assemble large, complex datasets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing datadelivery, re-designing infrastructure for greater scalability, etc with the guidance.
• Create datatools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Monitoring performance and advising any necessary infrastructure changes.
• Defining dataretention policies.
• Implementing the ETL process and optimal data pipeline architecture
• Build analytics tools that utilize the datapipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Create design documents that describe the functionality, capacity, architecture, and process.
• Develop, test, and implement datasolutions based on finalized design documents.
• Work with dataand analytics experts to strive for greater functionality in our data
• Proactively identify potential production issues and recommend and implement solutions

Skillsets:

• Good understanding of optimal extraction, transformation, and loading of datafrom a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Proficient understanding of distributed computing principles
• Experience in working with batch processing/ real-time systems using various open-source technologies like NoSQL, Spark, Pig, Hive, Apache Airflow.
• Implemented complex projects dealing with the considerable datasize (PB).
• Optimization techniques (performance, scalability, monitoring, etc.)
• Experience with integration of datafrom multiple data sources
• Experience with NoSQL databases, such as HBase, Cassandra, MongoDB, etc.,
• Knowledge of various ETL techniques and frameworks, such as Flume
• Experience with various messaging systems, such as Kafka or RabbitMQ
• Good understanding of Lambda Architecture, along with its advantages and drawbacks
• Creation of DAGs for dataengineering
• Expert at Python /Scala programming, especially for dataengineering/ ETL purposes

• Partnering with internal business owners (product, marketing, edit, etc.) to understand needs and develop custom analysis to optimize for user engagement and retention
• Good understanding of the underlying business and workings of cross functional teams for successful execution
• Design and develop analyses based on business requirement needs and challenges.
• Leveraging statistical analysis on consumer research and data mining projects, including segmentation, clustering, factor analysis, multivariate regression, predictive modeling, etc.
• Providing statistical analysis on custom research projects and consult on A/B testing and other statistical analysis as needed. Other reports and custom analysis as required.
• Identify and use appropriate investigative and analytical technologies to interpret and verify results.
• Apply and learn a wide variety of tools and languages to achieve results
• Use best practices to develop statistical and/ or machine learning techniques to build models that address business needs.

• 2 - 4 years  of relevant experience in Data science.
• Preferred education: Bachelor's degree in a technical field or equivalent experience.
• Experience in advanced analytics, model building, statistical modeling, optimization, and machine learning algorithms.
• Machine Learning Algorithms: Crystal clear understanding, coding, implementation, error analysis, model tuning knowledge on Linear Regression, Logistic Regression, SVM, shallow Neural Networks, clustering, Decision Trees, Random forest, XGBoost, Recommender Systems, ARIMA and Anomaly Detection. Feature selection, hyper parameters tuning, model selection and error analysis, boosting and ensemble methods.
• Strong with programming languages like Python and data processing using SQL or equivalent and ability to experiment with newer open source tools.
• Experience in normalizing data to ensure it is homogeneous and consistently formatted to enable sorting, query and analysis.
• Experience designing, developing, implementing and maintaining a database and programs to manage data analysis efforts.
• Experience with big data and cloud computing viz. Spark, Hadoop (MapReduce, PIG, HIVE).
• Experience in risk and credit score domains preferred.

GCP  Data Analyst profile must have below skills sets :

• Knowledge of programming languages like https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.simplilearn.com%2Ftutorials%2Fsql-tutorial%2Fhow-to-become-sql-developer&;data=05%7C01%7Ca_anjali%40hcl.com%7C4ae720b3f3cc45c3e04608da3346b335%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637878675987971859%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=EImfaJAD1KHOyrBQ7FkbaPl1STtfnf4QdQlbjw72%2BmE%3D&reserved=0" target="_blank">SQL, Oracle, R, MATLAB, Java and https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.simplilearn.com%2Fwhy-learn-python-a-guide-to-unlock-your-python-career-article&;data=05%7C01%7Ca_anjali%40hcl.com%7C4ae720b3f3cc45c3e04608da3346b335%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637878675987971859%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Z2n1Xy%2F3YN6nQqSweU5T7EfUTa1kPAAjbCMTWxDCh%2FY%3D&reserved=0" target="_blank">Python
• Data cleansing, data visualization, data wrangling
• Data modeling , data warehouse concepts
• Adapt to Big data platform like Hadoop, Spark for stream & batch processing
• GCP (Cloud Dataproc, Cloud Dataflow, Cloud Datalab, Cloud Dataprep, BigQuery, Cloud Datastore, Cloud Datafusion, Auto ML etc)

We are looking for an outstanding ML Architect (Deployments) with expertise in deploying Machine Learning solutions/models into production and scaling them to serve millions of customers. A candidate with an adaptable and productive working style which fits in a fast-moving environment.

Skills:

- 5+ years deploying Machine Learning pipelines in large enterprise production systems.

- Experience developing end to end ML solutions from business hypothesis to deployment / understanding the entirety of the ML development life cycle.
- Expert in modern software development practices; solid experience using source control management (CI/CD).
- Proficient in designing relevant architecture / microservices to fulfil application integration, model monitoring, training / re-training, model management, model deployment, model experimentation/development, alert mechanisms.
- Experience with public cloud platforms (Azure, AWS, GCP).
- Serverless services like lambda, azure functions, and/or cloud functions.
- Orchestration services like data factory, data pipeline, and/or data flow.
- Data science workbench/managed services like azure machine learning, sagemaker, and/or AI platform.
- Data warehouse services like snowflake, redshift, bigquery, azure sql dw, AWS Redshift.
- Distributed computing services like Pyspark, EMR, Databricks.
- Data storage services like cloud storage, S3, blob, S3 Glacier.
- Data visualization tools like Power BI, Tableau, Quicksight, and/or Qlik.
- Proven experience serving up predictive algorithms and analytics through batch and real-time APIs.
- Solid working experience with software engineers, data scientists, product owners, business analysts, project managers, and business stakeholders to design the holistic solution.
- Strong technical acumen around automated testing.
- Extensive background in statistical analysis and modeling (distributions, hypothesis testing, probability theory, etc.)
- Strong hands-on experience with statistical packages and ML libraries (e.g., Python scikit learn, Spark MLlib, etc.)
- Experience in effective data exploration and visualization (e.g., Excel, Power BI, Tableau, Qlik, etc.)
- Experience in developing and debugging in one or more of the languages Java, Python.
- Ability to work in cross functional teams.
- Apply Machine Learning techniques in production including, but not limited to, neuralnets, regression, decision trees, random forests, ensembles, SVM, Bayesian models, K-Means, etc.

Roles and Responsibilities:

Deploying ML models into production, and scaling them to serve millions of customers.

Technical solutioning skills with deep understanding of technical API integrations, AI / Data Science, BigData and public cloud architectures / deployments in a SaaS environment.

Strong stakeholder relationship management skills - able to influence and manage the expectations of senior executives.
Strong networking skills with the ability to build and maintain strong relationships with both business, operations and technology teams internally and externally.

Provide software design and programming support to projects.

 Qualifications & Experience:

Engineering and post graduate candidates, preferably in Computer Science, from premier institutions with proven work experience as a Machine Learning Architect (Deployments) or a similar role for 5-7 years.

We are looking for an outstanding Big Data Engineer with experience setting up and maintaining Data Warehouse and Data Lakes for an Organization. This role would closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Roles and Responsibilities:

• Develop and maintain scalable data pipelines and build out new integrations and processes required for optimal extraction, transformation, and loading of data from a wide variety of data sources using 'Big Data' technologies.
• Develop programs in Scala and Python as part of data cleaning and processing.
• Assemble large, complex data sets that meet functional / non-functional business requirements and fostering data-driven decision making across the organization.  
• Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems.
• Implement processes and systems to validate data, monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
• Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Provide high operational excellence guaranteeing high availability and platform stability.
• Closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Skills:

• Experience with Big Data pipeline, Big Data analytics, Data warehousing.
• Experience with SQL/No-SQL, schema design and dimensional data modeling.
• Strong understanding of Hadoop Architecture, HDFS ecosystem and eexperience with Big Data technology stack such as HBase, Hadoop, Hive, MapReduce.
• Experience in designing systems that process structured as well as unstructured data at large scale.
• Experience in AWS/Spark/Java/Scala/Python development.
• Should have Strong skills in PySpark (Python & SPARK). Ability to create, manage and manipulate Spark Dataframes. Expertise in Spark query tuning and performance optimization.
• Experience in developing efficient software code/frameworks for multiple use cases leveraging Python and big data technologies.
• Prior exposure to streaming data sources such as Kafka.
• Should have knowledge on Shell Scripting and Python scripting.
• High proficiency in database skills (e.g., Complex SQL), for data preparation, cleaning, and data wrangling/munging, with the ability to write advanced queries and create stored procedures.
• Experience with NoSQL databases such as Cassandra / MongoDB.
• Solid experience in all phases of Software Development Lifecycle - plan, design, develop, test, release, maintain and support, decommission.
• Experience with DevOps tools (GitHub, Travis CI, and JIRA) and methodologies (Lean, Agile, Scrum, Test Driven Development).
• Experience building and deploying applications on on-premise and cloud-based infrastructure.
• Having a good understanding of machine learning landscape and concepts. 

Qualifications and Experience:

Engineering and post graduate candidates, preferably in Computer Science, from premier institutions with proven work experience as a Big Data Engineer or a similar role for 3-5 years.

Certifications:

Good to have at least one of the Certifications listed here:

    AZ 900 - Azure Fundamentals

    DP 200, DP 201, DP 203, AZ 204 - Data Engineering

    AZ 400 - Devops Certification

We are looking for passionate, talented and super-smart engineers to join our product development team. If you are someone who innovates, loves solving hard problems, and enjoys end-to-end product development, then this job is for you! You will be working with some of the best developers in the industry in a self-organising, agile environment where talent is valued over job title or years of experience.

Responsibilities:

• You will be involved in end-to-end development of VIMANA technology, adhering to our development practices and expected quality standards.
• You will be part of a highly collaborative Agile team which passionately follows SAFe Agile practices, including pair-programming, PR reviews, TDD, and Continuous Integration/Delivery (CI/CD).
• You will be working with cutting-edge technologies and tools for stream processing using Java, NodeJS and Python, using frameworks like Spring, RxJS etc.
• You will be leveraging big data technologies like Kafka, Elasticsearch and Spark, processing more than 10 Billion events per day to build a maintainable system at scale.
• You will be building Domain Driven APIs as part of a micro-service architecture.
• You will be part of a DevOps culture where you will get to work with production systems, including operations, deployment, and maintenance.
• You will have an opportunity to continuously grow and build your capabilities, learning new technologies, languages, and platforms.

Requirements:

• Undergraduate degree in Computer Science or a related field, or equivalent practical experience.
• 2 to 5 years of product development experience.
• Experience building applications using Java, NodeJS, or Python.
• Deep knowledge in Object-Oriented Design Principles, Data Structures, Dependency Management, and Algorithms.
• Working knowledge of message queuing, stream processing, and highly scalable Big Data technologies.

• Experience in working with Agile software methodologies (XP, Scrum, Kanban), TDD and Continuous Integration (CI/CD).
• Experience using no-SQL databases like MongoDB or Elasticsearch.
• Prior experience with container orchestrators like Kubernetes is a plus.

• We recognize that our dedicated team members make us successful and we offer competitive salaries.
• We are a workplace that values work-life balance, provides flexible working hours, and full time remote work options.
• You will be part of a team that is highly motivated to learn and work on cutting edge technologies, tools, and development practices.
• Bon Appetit! Enjoy catered breakfasts, lunches and free snacks!