Вакансия


Senior data engineer
Опубликовано 21 октября
Senior data engineer
ООО ClearScale
Город
Санкт-Петербург
Обязанности

ABOUT CLEARSCALE
ClearScale is a leading cloud consulting company that provides a wide range of services including cloud architecture design, application development, system integration, cloud migration, automation, and managed services. We hold a prestigious AWS Premier Consulting partner status and help Fortune 500 enterprises, mid-sized businesses, and startups succeed with ambitious, challenging, and unique cloud projects. We architect, develop, and launch innovative and sophisticated solutions using the best cutting-edge cloud technologies. ClearScale is growing quickly and there is a high demand for the services we provide. Clients come to us for our deep experience in cloud infrastructure, migrations, DevOps, and automation
About the role
ClearScale is looking for a generalist Data Engineer to participate in a large-scale cloud migration project. We are hiring for a 2-year-long project related to a migration from a 3rd party cloud to AWS, including applications re-factoring to microservice architecture and using cloud-native AWS services. You will be part of the distributed system engineering team, responsible for preparing infrastructure, re-hosting applications, re-designing, load testing, migration of high-load applications, workload optimization, and operational support. This is a mass migration led by ClearScale with thousands of servers on the inventory list. The technologies include Kubernetes (EKS), Redshift, Kinesis, Kinesis Analytics, Aurora, DocumentDB, DynamoDB, DB2, Oracle, Mongo, Hadoop/Spark, MSSQL, IIS, Kafka, Elasticsearch, Solr, Redis, Kubernetes, and a few more. Responsibilities

- Analyze, scope and estimate tasks, identify technology stack and tools
- Design and implement optimal architecture and migration plan
- Specify the infrastructure and assist DevOps engineers with implementing the infrastructure
- Examine performance and advise necessary infrastructure changes
- Communicate with client on data-related issues and support data infrastructure needs
- Collaborate with in-house and external development and analytical teams

Qualifications
ClearScale expects successful candidate to have most of the following qualifications and skills (not necessary all have to be presented):
- We are looking for a candidate with 5+ years of experience in a Data or Software Engineer role, who has attained a degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. - Hands-on experience building and optimizing ‘big data’ architectures and data pipelines
- Hand-on experience designing efficient data architectures for high-load enterprise-scale applications
- Successful candidates should have experience with some of the following software/tools: - Big data tools: Kafka, Spark, Hadoop, Hive
- Relational SQL and NoSQL databases, including PostgreSQL, Cassandra, Couchbase, MongoDB and Elasticsearch
- Message queue systems: ActiveMQ, RabbitMQ, AWS SQS, etc
- Data pipeline and workflow management tools: AWS Glue, Airflow, NiFi, Step Functions
- AWS cloud services: Kinesis, Kinesis Analytics, EMR, RDS, Redshift, DocumentDB, Lambda
- Stream-processing systems: Kinesis Streaming, Spark-Streaming, Kafka Streams
- Object-oriented/object function scripting languages: Java, Scala, Python, Go, C++


- Build processes supporting data transformation, data structures, metadata, dependency and workload management
- Hands-on experience with message queuing, stream processing and highly scalable ‘big data’ data stores
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
- Strong analytic skills related to working with unstructured documentation and datasets
- Strong self-management skills
- English language frequent user - you should be able to commit, write and speak to the team/client

Will be a huge plus:
- DB: Oracle, DB2
- Working knowledge of commercial BI and data warehousing tools such as Tableau, Snowflake or Pentaho
- Remote working experience would be good but not essential

Valid AWS certificates would be a great plus (not a must):
- AWS Certified Solutions Architect – (Associate / Professional)
- AWS Certified DevOps Engineer – (Associate / Professional)
- AWS Certified SysOps Administrator – (Associate)
- AWS Certified Developer – (Associate)

You’ll be a great fit if:
- You'd like to work remotely with a flexible schedule
- You thrive in a small, dynamic, and agile team that encourages you to learn and grow
- You desire to work with some of the world’s top brands
- You enjoy finding solutions to interesting problems and figuring out how things work
- You welcome having autonomy with complex tasks
- You are passionate about using your experience and expertise to inspire the team

Требования
Пол: Не важно


Поделитесь с друзьями
Похожие вакансии



Rabotunaidu
г. Серпухов, Московское ш., д. 40 Россия
rabotunaidu.ru