з/п не указана
Lamoda, tech.lamoda.ru - is a complex highload project, almost 300 specialists work on it, including developers, QAs, analysts, system administrators, product managers and designers. We develop, test and support over 60 services. These are systems that automatize processes of the fulfillment center, contact centers, photo studio, delivery service and other operational systems. Apart from that we create high-loaded services: website, mobile API, catalogue, personalization and obviously develop website and mobile applications front-end.
What you’ll be doing:
- Design and build next generation best on market e-commerce data platform
- Build and maintain reliable and scalable enterprise level distributed transactional data processing systems for scaling the existing business and supporting new business initiatives
- Optimize jobs to utilize Kafka, Hadoop, Spark resources in the most efficient way
- Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
- Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build/deploy tools and datasets that fit their use cases)
- Perform code review
- Provide mentorship and guidance to junior team members
- Performance tuning, monitoring, capacity planning of Hadoop cluster and other infrastructure
- Ingest, validate and process internal & third party data
- Create, maintain and monitor data flows for consistency, accuracy and lag time
- Maintain and enhance framework for jobs (Spark Streaming, Spark, MapReduce, aggregate jobs in Hive)
- Train Developers/Analysts on tools to pull data
- Tool evaluation/selection/implementation
- Review/Approval - DDL for database, Hive Framework jobs and Spark Streaming to make sure hey meet our standards
- BA/BS degree in Computer science or related field
- Experience in Hadoop stack (YARN, HDFS, HBase, Hive)
- Experience in Spark
- Experience in Kafka
- Experience in Docker, Kubernetes is a plus
- Knowledge of version control systems (git)
- Experience in Python is a huge plus
- Strong understanding of RDBMS, SQL;
- Passion for engineering and computer science around data
What can you expect?
- Challenging and interesting atmosphere in a young team of professionals;
- Official employment, social package;
- Comfortable office in city center m. Polezaevskaya / m. Horoshevskaya / mck Horoshevo.
- Competitive salary;
- Free English courses;
- Discount on the entire range of products.
- High exposure to different departments, data sources and business problems.
- Collaboration and discussion with our technology partners (e.g. Facebook, Google, Yandex, VK, Amazon).
- Exposure to the newest technologies in the data science space.