Big Data Engineer

Company: Synchrony
Location: Walnut Creek
Posted on: March 16, 2023

Job Description:

Job Description:**Role Summary/Purpose:** The candidate Big Data Engineer will join an Agile scrum team and perform functional & system development for Synchrony's Enterprise Data Lake. As a Big Data Engineer the ability to integrate data across internal and external sources, provide analytical insights, and integrate with our critical systems are key skills. The engineer will participate in data analysis efforts to ensure the delivery of high-quality data ingestion, standardization and curation and maintain compliance with the applicable Data Sourcing, Data Quality, and Data Governance standards. The engineer will drive quality through the entire software development lifecycle with focus on functional requirements, efficiency, and methodology. The engineer will work cross-functionally with operations, other data engineers and product owner to assure capabilities are delivered that meet business needs. We're proud to offer you choice and flexibility. You have the option to be remote, and work from home, or come into one of our offices. You may be occasionally requested to commute to our nearest office for in person engagement activities such as team meetings, training and culture events.**Essential Responsibilities:** + Develop big data applications for Synchrony in Hadoop ecosystem+ Participate in the agile development process including backlog grooming, coding, code reviews, testing and deployment+ Work with team members to achieve business results in a fast paced and quickly changing environment+ Work independently to develop analytic applications leveraging technologies such as: Hadoop, NoSQL, In-memory Data Grids, Kafka, Spark, Ab Initio+ Provide data analysis for Synchrony's data ingestion, standardization and curation efforts ensuring all data is understood from a business context+ Identify enablers and level of effort required to properly ingest and transform data for the data lake.+ Profile data to assist with defining the data elements, propose business term mappings, and define data quality rules+ Work with the Data Office to ensure that data dictionaries for all ingested and created data sets are properly documented in data dictionary repository+ Ensure the lineage of all data assets are properly documented in the appropriate enterprise metadata repositories+ Assist with the creation and implementation of data quality rules+ Ensure the proper identification of sensitive data elements and critical data elements+ Create source-to-target data mapping documents+ Test current processes and identify deficiencies+ Investigate program quality to make improvements to achieve better data accuracy+ Understand functional and non-functional requirement and prepare test data accordingly+ Plan, create and manage the test case and test script+ Identify process bottlenecks and suggest actions for improvement+ Execute test script and collect test results+ Present test cases, test results, reports and metrics as required by the Office of Agile+ Perform other duties as needed to ensure the success of the team and application and ensure the team's compliance with the applicable Data Sourcing, Data Quality, and Data Governance standards+ Perform other duties and/or special projects as assigned**Qualifications/Requirements:** + Bachelor's degree in a quantitative field (such as Engineering, Computer Science, Statistics, Econometrics); in lieu of degree, High School Diploma/GED and minimum 2 years of Information Technology experience+ Hands-on experience writing shell scripts, complex SQL queries, Hive scripts, Hadoop commands and Git+ Ability to write abstracted, reusable code components+ Programming experience in at least one of the following languages: Scala, Java or Python+ Analytical mindset+ Willingness and aptitude to learn new technologies quickly+ Superior oraland written communication skills;+ Ability to collaborate across teams of internal and external technical staff, business analysts, software support and operations staff. **Desired Characteristics:** + Performance tuning experience+ Exposure to the following Ab Initio tools: GDE - Graphical Development Environment; Co> Operating System ; Control Center; Metadata Hub; Enterprise Meta> Environment; Enterprise Meta> Environment Portal; Acquire> It; Express> It; Conduct> It; Data Quality Environment; Query> It.+ Familiar with Ab Intio, Hortonworks/Cloudera, Zookeeper, Oozie and Kafk+ Familiar with Public Cloud (i.e. AWS, GCP, Azure) data engineering services+ Familiar with data management tools (i.e. Collibra)+ Background in ETL, data warehousing or data lake+ Strong business acumen including a broad understanding of Synchrony business processes and practices+ Demonstrated ability to work effectively in an agile team environment+ Financial Industry or Credit processing experience+ Experience with working on a geographically distributed team managing onshore/offshore resources with shifting priorities+ Previous experience working in?client facing environment+ Proficient in the maintenance of data dictionaries and other information in Collibra+ Excellent analytical, organizational and influencing skills with a proven track record of successfully executing on assignments and initiatives**Grade/Level: 09** The salary range for this position is **56,000.00 - 112,000.00** USD AnnualSalaries are adjusted according to market in CA and Metro NY and some positions are bonus eligible.**Eligibility Requirements:** + You must be 18 years or older+ You must have a high school diploma or equivalent+ You must be willing to take a drug test, submit to a background investigation and submit fingerprints as part of the onboarding process+ You must be able to satisfy the requirements of Section 19 of the Federal Deposit Insurance Act.+ New hires (Level 4-7) must have 9 months of continuous service with the company before they are eligible to post on other roles. Once this new hire time in position requirement is met, the associate will have a minimum 6 months' time in position before they can post for future non-exempt roles. Employees, level 8 or greater, must have at least 24 months' time in position before they can post. All internal employees must consistently meet performance expectations and have approval from your manager to post (or the approval of your manager and HR if you don't meet the time in position or performance expectations).Legal authorization to work in the U.S. is required. We will not sponsor individuals for employment visas, now or in the future, for this job opening. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status. **Our Commitment:** When you join us, you'll be part of a diverse, inclusive culture where your skills, experience, and voice are not only heard-but valued. We celebrate the differences in all of us and believe that our individual, unique perspectives is what makes Synchrony truly a great place to work. Together, we're building a future where we can all belong, connect and turn ideals into action. Through the power of our 8 Diversity Networks+ (https://www.synchronycareers.com/our-culture/#diversity) , with more than 60% of our workforce engaged, you'll find community to connect with an opportunity to go beyond your passions.This starts when you choose to apply for a role at Synchrony. We ensure all qualified applicants will receive consideration for employment without regard to age, race, color, religion, gender, sexual orientation, gender identity, national origin, disability, or veteran status.**Reasonable Accommodation Notice:** + Federal law requires employers to provide reasonable accommodation to qualified individuals with disabilities. Please tell us if you require a reasonable accommodation to apply for a job or to perform your job. Examples of reasonable accommodation include making a change to the application process or work procedures, providing documents in an alternate format, using a sign language interpreter, or using specialized equipment.+ If you need special accommodations, please call our Career Support Line so that we can discuss your specific situation. We can be reached at 1-866-301-5627. Representatives are available from 8am - 5pm Monday to Friday, Central Standard Time**Job Family Group:** Information Technology

Keywords: Synchrony, Walnut Creek , Big Data Engineer, Other , Walnut Creek, California