Data Engineer - PySpark

Data Engineer - PySpark

Zaměření: Analytics and Emerging Digital Technologies
Město: London, England - London, United Kingdom
ID pozice: J0221-2960
Typ pracovního poměru: Plný úvazek

Popis pozice:

If your programming skills are matched by an enthusiasm for problem-solving, and your communication skills are as creative as your coding, there’s a digital revolution underway in the UK that could use your help.

Our Financial Services business unit is looking to add a couple of talented PySpark Developers to a programme of work with one of or key Investment Banking clients.


From public service transformation to cutting-edge commercial projects, our work has a wide reach, helping make the everyday easier, for everyone. When you start your CGI story, not only will you be improving people’s lives, you can be sure we’ll do the same for you.

Alongside a competitive salary, we offer you access to our share scheme (3% + 3% matching) making you a member not just an employee. You also have the option to benefit from private medical and dental insurance, flexible retirement options, childcare vouchers and a cycle to work scheme.
As for the future? We work as one team, and are a Top 100 employer. Be a successful part of our story and we’ll support you in taking your career anywhere you want it to go.

This role can be based out of our London office as well as other CGI offices.

O čem tato pozice je:

Working on a large programme of work within an Investment you will have a number of responsibilities including:
• Contributes to quality assurance by writing unit and functional tests.
• Ensures development happens for all Software Components in accordance with Detailed Software Requirements specification, the functional design and the technical design document.
• Designing and writing code in accordance with provided business requirements
• Analysing and performance tuning Spark queries e.g. looking at the DAG
• You will be developing source code, including descriptions in line with detailed requirements, then verifying the developed source code by reviews in addition to measuring and analysing code for quality
• Integrating software components including unit and integrated software testing according to the test plan
• Ensuring that all code changes, end up in configuration and deployment tools
• Providing Release Deployments on non-Production Management controlled environments
• Providing Level 3 support for technical components and contributing to problem and root cause analysis
• Collaborating with Functional Analysts and technical Specialists to complete work

Co od Tebe očekáváme:

• You must have excellent experience with the Fundamentals of Spark using the Dataframe API
• Understanding partitioning of data
• Experience of analysing and performance tuning Spark queries e.g. looking at the DAG
• Knowledge of Hadoop and its ecosystem of technologies especially Hive
• Python and OOP concepts using Python is essential
• Knowledge of Conditional Statements & Loops: If-else Control Structures, For/While Loops
• Demonstrate a comprehensive understanding of Complex Data Types: Shallow & Deep Copies, Working with Lists & Tuples, Dictionaries & Sets
• Understand Fundamental Data Structures & their Implementation
• Good knowledge of Exceptions & Command Line Arguments
• Basic knowledge of UNIX
• Are able to demonstrate source control knowledge (preferably GIT)
• Ability to analyse databases directly using query language tools such as SQL
• Experience on ETL process on Big Data
• Have an understanding of data relationships, normalisation

Non-Technical
• Use of JIRA / Confluence
• Appreciation of release management and software maintenance
• Strong analytical skills.
• Should have good understanding of architecture
• Ability to work in virtual teams
• An Excellent team player and open minded approach
• Ability to share information, transfer knowledge and expertise to team members.
An Ability to work in a fast paced environment with competing and alternating priorities with constant focus on delivery.
• Candidate is expected to have high desire to learn new technologies and implement various solutions in fast paced environment.

Desirable experience:
• Knowledge of control M
• Hands on experience of Team City
• Understanding of Financial Products
• Ability to interpret and write complex SQLs

Od můžeš očekávat Ty od nás:

Build your career with us.

It is an extraordinary time to be in business. As digital transformation continues to accelerate, CGI is at the center of this change—supporting our clients’ digital journeys and offering our professionals exciting career opportunities.

At CGI, our success comes from the talent and commitment of our professionals. As one team, we share the challenges and rewards that come from growing our company, which reinforces our culture of ownership. All of our professionals benefit from the value we collectively create.

Be part of building one of the largest independent technology and business services firms in the world.

Learn more about CGI at www.cgi.com.

No unsolicited agency referrals please.

CGI is an equal opportunity employer.