Data Engineer – Web Scraping & Data Acquisition

Trigyn has a contractual opportunity as Data Engineer – Web Scraping & Data Acquisition. This resource will be working at the client site in in Lebanon. Job Description: • Design, develop, and maintain ETL/ELT pipelines for automated data collection, transformation, and integration. • Develop and maintain web scraping and data extraction solutions to collect structured and unstructured data from national, regional, and international sources.

• Automate data ingestion from multiple sources, including websites, APIs, databases, files, and other external data services. Implement data validation, cleansing, quality checks, and monitoring mechanisms to ensure the accuracy, completeness, and reliability of collected data. • Develop resilient scraping processes capable of handling changes in source websites, data structures, pagination, and other extraction challenges.

• Utilize Azure and other relevant platforms and technologies to support scalable data processing, storage, automation, and deployment. • Collaborate with Data Scientists, Data Analysts, and Full-Stack Web Developers to provide clean, structured, and reliable datasets for applications, analytics, and reporting. • Prepare and maintain technical documentation, including data-source specifications, pipeline and scraper documentation, operational procedures, functional specifications, and relevant training materials.

• Provide ongoing support and maintenance for deployed data pipelines and web scrapers, including troubleshooting, version control, monitoring, data recovery, and deployment management. • Research emerging technologies and industry trends, staying up to date with developments in the field, and actively assisting in testing and evaluating new products, tools, and technologies. Qualifications A minimum high school diploma with a certification or degree in Information Systems, Information Technology, Computer Science, Statistics or related area is required.

• A minimum of 4 years of professional experience with Python, SQL, cloud environments (Azure) and DevOps automation is required. • Demonstrated experience in developing and maintaining web scraping and automated data extraction solutions, including handling structured and unstructured web-based data sources, is required. • Experience with data validation, transformation, quality assurance, monitoring, and troubleshooting of automated data pipelines is required.

• Excellent analytical, problem-solving, documentation, and communication skills, with the ability to collaborate effectively with multidisciplinary technical teams, is required. • Knowledge in database design, and development using MS SQL Relational Database is required. For Immediate Response, Please send your Resume to [email protected] TRIGYN TECHNOLOGIES is a multinational IT services company with resources deployed in 28 countries.

TRIGYN is an ISO 9001:2015, ISO 27001:2013 (ISMS) and CMMI Level 5 certified company. TRIGYN has offices in the United States, Canada, Switzerland and India.