Senior data engineer (aws)
Опубликовано 20 сентября
Senior data engineer (aws)
ООО ClearScale


ClearScale is a leading cloud consulting company that provides a wide range of services including cloud architecture design, application development, system integration, cloud migration, automation, and managed services. We hold a prestigious AWS Premier Consulting partner status and help Fortune 500 enterprises, mid-sized businesses, and startups succeed with ambitious, challenging, and unique cloud projects. We architect, develop, and launch innovative and sophisticated solutions using the best cutting-edge cloud technologies. ClearScale is growing quickly and there is a high demand for the services we provide. Clients come to us for our deep experience in cloud infrastructure, migrations, DevOps, and automation
About the role
ClearScale is looking for a generalist Data Engineer to participate in a large-scale cloud migration project. We are hiring for a 2-year-long project related to a migration from a 3rd party cloud to AWS, including applications re-factoring to microservice architecture and using cloud-native AWS services. You will be part of the distributed system engineering team, responsible for preparing infrastructure, re-hosting applications, re-designing, load testing, migration of high-load applications, workload optimization, and operational support. This is a mass migration led by ClearScale with thousands of servers on the inventory list. The technologies include Kubernetes (EKS), Redshift, Kinesis, Kinesis Analytics, Aurora, DocumentDB, DynamoDB, DB2, Oracle, Mongo, Hadoop/Spark, MSSQL, IIS, Kafka, Elasticsearch, Solr, Redis, Kubernetes, and a few more. Responsibilities
· Analyze, scope and estimate tasks, identify technology stack and tools
· Design and implement optimal architecture and migration plan
· Specify the infrastructure and assist DevOps engineers with implementing the infrastructure
· Examine performance and advise necessary infrastructure changes
· Communicate with client on data-related issues and support data infrastructure needs
· Collaborate with in-house and external development and analytical teams
ClearScale expects successful candidate to have most of the following qualifications and skills (not necessary all have to be presented): · We are looking for a candidate with 5+ years of experience in a Data or Software Engineer role, who has attained a degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. · Hands-on experience building and optimizing ‘big data’ architectures and data pipelines
· Hand-on experience designing efficient data architectures for high-load enterprise-scale applications
· Successful candidates should have experience with some of the following software/tools: a. Big data tools: Kafka, Spark, Hadoop, Hive
b. Relational SQL and NoSQL databases, including PostgreSQL, Cassandra, Couchbase, MongoDB and Elasticsearch
c. Message queue systems: ActiveMQ, RabbitMQ, AWS SQS, etc
d. Data pipeline and workflow management tools: AWS Glue, Airflow, NiFi, Step Functions
e. AWS cloud services: Kinesis, Kinesis Analytics, EMR, RDS, Redshift, DocumentDB, Lambda
f. Stream-processing systems: Kinesis Streaming, Spark-Streaming, Kafka Streams
g. Object-oriented/object function scripting languages: Java, Scala, Python, Go, C++
· Build processes supporting data transformation, data structures, metadata, dependency and workload management
· Hands-on experience with message queuing, stream processing and highly scalable ‘big data’ data stores
· Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
· Working knowledge of commercial BI and data warehousing tools such as Tableau, Snowflake or Pentaho would be a plus
· Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
· Strong analytic skills related to working with unstructured documentation and datasets
· Strong self-management and self-organizational skills
Will be a huge plus:
- DB: Oracle, DB2
- Remote working experience would be good but not essential

Valid AWS certificates would be a great plus (not a must): · AWS Certified Solutions Architect – (Associate / Professional)
· AWS Certified DevOps Engineer – (Associate / Professional)
· AWS Certified SysOps Administrator – (Associate)
· AWS Certified Developer – (Associate)
You’ll be a great fit if:
- You'd like to work remotely with a flexible schedule
- You thrive in a small, dynamic, and agile team that encourages you to learn and grow
- You desire to work with some of the world’s top brands
- You enjoy finding solutions to interesting problems and figuring out how things work
- You welcome having autonomy with complex tasks
- You are passionate about using your experience and expertise to inspire the team

Пол: Не важно

Поделитесь с друзьями

г. Серпухов, Московское ш., д. 40 Россия