Menu

Data Engineer

Explorium is on a mission to transform the way organizations use data to build their unique competitive advantage.

We are redefining how companies find relevant data for AI by automatically discovering the most predictive datasets and features across thousands of sources, on the web and in the enterprise. The Explorium Data Science Platform tackles the toughest challenges in the predictive model building process – data integration, data enrichment and feature engineering – enabling data specialists to build machine learning models based on the best data possible. Combining their own proprietary data assets with the best external sources in the world is the competitive edge organizations need to build the predictive models that will drive their future growth.

Job Description

As a Web Scraping focused Data Engineer, you will be responsible for extracting and ingesting data from websites using web crawling tools. In this role you will own the creation process of these tools, services, and workflows to improve crawl/ scrape analysis, reports and data management. We will rely on you to test the data and the scrape to insure accuracy and quality. You will own the process to identify and rectify any issues with breaks as well as scale scrapes as needed.

Skills And Qualifications

  • Experience running large scale web scrapes
  • Solid Python knowledge
  • Familiarity with Linux/UNIX, HTTP, HTML, Javascript and Networking
  • Familiarity with techniques and tools for crawling, extracting and processing data (e.g. Scrapy, pandas, mapreduce, SQL, BeautifulSoup, etc)
  • Experience with version control, open source practices, and code review
  • Great communication skills (written and Spoken in English)
  • Bachelor’s Degree in Computer Science or a related field or the equivalent demonstrated experience
  • Experience with system monitoring/administration tools – advantage
Back to open roles

Apply for this role







Attach resume
Attach Cover Letter

Please review our privacy policy

This site uses cookies to provide you with a great browsing experience. By continuing on our website, you accept our cookies policy. I accept