Senior Data Engineer
Hello, Hope you are Doing good, Position:
Senior Data Engineer Location:
Jersey city, NJ (first preference) Other is Plano, TX and Wilmington, DE (only local candidate) Duration:
Contract opportunity
Job Description:
About Project:
Any 3 days in office is must should be local to (NJ, TX or DE only) Liquidity Risk Reporting - fulfill Fed obligation - daily reporting Stress Scenario report - generate report based on multiple scenarios then generate into Living Will of the firm - critical report.
Deliverables in June and September Infra is on-prem Hadoop - this will be delivered on AWS.
Must have:
Spark, Scala, Java, AWS (EMR) Nice to have:
Python, S3 EC2, EKS, Hadoop, Impala, Hive, Airflow Domain knowledge a plus Data set size is 500m-1b rows per batch.
Interviews - 2 rounds focused on technical - Scala, Spark, write code, answer design questions.
Job
Responsibilities:
Developed and maintained end-to-end data pipelines using Python, Scala, and Spark for processing and analysing large volumes of data related to Market Analysis, Fraud Detection, Customer Segmentation, and Data Security and Privacy.
Implemented Hadoop, HBase, and Hive for scalable and optimized data storage and retrieval, enabling efficient handling and analysis of data.
Collaborated with cross-functional teams, including data scientists, software engineers, and business stakeholders, to gather requirements and deliver high-impact machine learning solutions in batch processing environments.
Researched and adopted cutting-edge data engineering and machine learning technologies to stay abreast of industry trends and continuously improve the performance of batch processing systems.
Utilized version control and automated testing practices to maintain the integrity and reproducibility of the batch-processing pipelines in collaboration with the data science team.
Utilized AWS services, such as S3, EMR, and EC2, to build robust and scalable big data solutions, ensuring high availability and data processing capabilities.
Engineered data transformation processes, including feature engineering and dimensionality reduction techniques, to optimize data representation for machine learning algorithms.
Managed data ingestion and extraction processes using Sqoop, Flume, and Kafka, enabling seamless integration with various data sources and real-time data streaming.
Viraj Sr.
Technical Recruiter Neumeric Technologies Recommended Skills Apache Flume Apache H Base Apache Hadoop Apache Hive Apache Kafka Apache Spark Estimated Salary: $20 to $28 per hour based on qualifications.
Senior Data Engineer Location:
Jersey city, NJ (first preference) Other is Plano, TX and Wilmington, DE (only local candidate) Duration:
Contract opportunity
Job Description:
About Project:
Any 3 days in office is must should be local to (NJ, TX or DE only) Liquidity Risk Reporting - fulfill Fed obligation - daily reporting Stress Scenario report - generate report based on multiple scenarios then generate into Living Will of the firm - critical report.
Deliverables in June and September Infra is on-prem Hadoop - this will be delivered on AWS.
Must have:
Spark, Scala, Java, AWS (EMR) Nice to have:
Python, S3 EC2, EKS, Hadoop, Impala, Hive, Airflow Domain knowledge a plus Data set size is 500m-1b rows per batch.
Interviews - 2 rounds focused on technical - Scala, Spark, write code, answer design questions.
Job
Responsibilities:
Developed and maintained end-to-end data pipelines using Python, Scala, and Spark for processing and analysing large volumes of data related to Market Analysis, Fraud Detection, Customer Segmentation, and Data Security and Privacy.
Implemented Hadoop, HBase, and Hive for scalable and optimized data storage and retrieval, enabling efficient handling and analysis of data.
Collaborated with cross-functional teams, including data scientists, software engineers, and business stakeholders, to gather requirements and deliver high-impact machine learning solutions in batch processing environments.
Researched and adopted cutting-edge data engineering and machine learning technologies to stay abreast of industry trends and continuously improve the performance of batch processing systems.
Utilized version control and automated testing practices to maintain the integrity and reproducibility of the batch-processing pipelines in collaboration with the data science team.
Utilized AWS services, such as S3, EMR, and EC2, to build robust and scalable big data solutions, ensuring high availability and data processing capabilities.
Engineered data transformation processes, including feature engineering and dimensionality reduction techniques, to optimize data representation for machine learning algorithms.
Managed data ingestion and extraction processes using Sqoop, Flume, and Kafka, enabling seamless integration with various data sources and real-time data streaming.
Viraj Sr.
Technical Recruiter Neumeric Technologies Recommended Skills Apache Flume Apache H Base Apache Hadoop Apache Hive Apache Kafka Apache Spark Estimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
