Learnings
Advanced Git concepts and their workflows.
CI/CD tools used to manage software development.
Build knowledge on top of my existing one in python.
Test Driven Development.
Managing huge datasets efficiently using spark, HDFS.
Learned how to develop projects within team environment.
Technologies
GIT version control system.
Object Oriented Programming.
Python including various modules such as Pandas, PySpark, Pytest, PyHDFS etc.
Jira, Jenkins
Worked on file systems like NFS, HDFS.
Contributions
Overhauled many python scripts for reporting service as per the clients/internal requirements.
Implemented python scripts with complex SQL queries to performed various operations on huge datasets from ODS using pyspark.
Designed code to replicate data files to Large file store so that we don't lose data even if in case of any mishap.
Created and merged over 30 pull requests during my internship period to the Bloomberg's existing codebase.
All the scripts on which I worked are in production and are being used in the company.