Data Engineer (Hadoop, Spark) Plano, TX

Company: AgreeYa Solutions
Location: Plano
Posted on: May 21, 2020

Job Description:

Title Hadoop Spark Developer Location Plano, TX Duration 18 Months (Extendable) Responsible for designs, develops, modifies, debugs andor maintains software systems, Serves as an expert on specific modules, applications or technologies, and deals with complex assignments during the software development life cycle. What will your job look like? You will take ownership and accountability of specific modules within an application and provides technical support and guidance during solution design for new requirements, problem resolution for critical complex issues You will Ensures code is maintainable, scalable and supportable. You will present demos of the software products to stakeholders and internalexternal customers, using knowledge of the productsolution and technologies to influence the direction and evolution of the productsolution. You will investigate issues by reviewingdebugging code and providing fixes (analyzes and fixes bugs) and workarounds, will review changes for operability to maintain existing software solutions, will highlight risks and will help mitigate risks from technical aspects. You will bring continuous improvementsefficiencies to the software or business processes by utilizing software engineering tools and various innovative techniques, and reusing existing solutions. By means of automation, reduces design complexity, reduces time to response, and simplifies the clientend-user experience. You will representlead discussions related to productapplicationmodulesteam (for example, leads technical design reviews). Builds relationships with internal customersstakeholders All you need is Bachelor's or Masters degree in ScienceITComputingBusiness Analytics or equivalent. 5+ years total experience in development mainly around Bigdata, Hive Hadoop, Spark, Scala, PythonPyspark, AWS, and other cloud related technologies. Experienced in Data Engineering with good understanding of Datawarehouse, Data Lake, Data Modelling, Parsing, Data wrangling, Cleansing Transformation, and sanitizing. Agile work experience, build CICD pipelines using Jenkins Hands-on Development experience with Scala, Python using Spark 2.0, Spark Internals and Spark jobs performance improvement Good understanding of Yarn, Spark UI, Spark resource management and Hadoop resource management and efficient Hadoop storage mechanisms. Good understanding experience with Performance tuning in Cloud environment for complex SW projects mainly around large scale and low latency. AWS knowledge is essential with good working experience in AWS Technologies EMR, S3, Cluster management, AWS Airflow automation. Snowflake Knowledge is plus. AWS development certificationSpark certifications is an advantage

Keywords: AgreeYa Solutions, Plano , Data Engineer (Hadoop, Spark) Plano, TX, Engineering , Plano, Texas