Must-have skills:
Nice-to-have skills:
Considering candidates from:
Europe
Europe
Work arrangement: Onsite
Industry: Software Development
Language: English
Level: Middle
Required experience: 2+ years
Relocation: Not paid
Visa support: Not provided
Size: 201 - 500 employees
Big Data Engineer
Software Development company
Barcelona, Spain
Description
We are looking for a Data Engineer for the largest and fastest-growing marketplace to buy and sell locally. With over 100M downloads and hundreds of millions of listings to date, they've been named among the “Hottest Startups” by Wired and “Best Apps” by Google, and it’s the second-fastest growing app in America since its 2015 launch. Based in New York and Barcelona offices, the company’s goal is to build the largest marketplace of its kind in the world. Note: This job is only for candidates who have a Spanish or EU work permit. Your challenge:
- Collect and process data from hundreds of sources daily (half a billion events of hundreds of types being produced daily)
- Collaborate with Data Scientists to develop experiments and deploy solutions to production
- Curate massive amounts of data and make it accessible
- Design highly scalable, concurrent and robust data architectures
- Ensure product and technical features are delivered to spec and on-time
- Design and implement features in collaboration with product owners, reporting analysts/data analysts, and business partners within an Agile / Scrum methodology
- Streaming processing tools like Kafka / Kinesis, Storm, Flink, Spark Streaming, etc
- Storage of data with any of these: Cassandra, HBase, Elastic Search, Solr, Redshift, any NoSQL or graph databases
- Querying tools: Hive, Presto, Drill, Impala, SparkSQL, etc.
- Batch processing with Hadoop or Spark
Desirable skills:
- Concurrent and distributed programming: reactive extensions, AKKA, etc
- Graph processing: Spark GraphX, Apache Giraph
- Lambda architecture, kappa architecture
Ideal skills:
- YARN, Kubernetes, Docker
- Cloudera, MapR knowledge
- Machine Learning knowledge
- Sci-py, R, Scala
- Notebook (Jupiter, Zeppelin, Databricks)
- SparkML, Deeplearning4j, H2O, Tensorflow, etc
Perks:
- 100% covered medical, vision, and dental care
- Competitive salary and stock options
- Beautiful, new office in the heart of Soho and along the beach in Barcelona
- Potential opportunities to travel to - and work in - both of our offices
- Robust relocation package
- Massage therapy (Barcelona office)
- Fully stocked kitchens
- Team-building activities
- Language and other courses