Web Developer

Context

Techno Import Distribution is a company specializing in the sale of vinyl records (electronic music). Its activity relies entirely on its e-commerce site. Upon my arrival, the company faced a major bottleneck regarding its catalog management: new products had to be added manually, an extremely time-consuming process for the sales teams.

My Mission

The goal of my mission was to digitize and accelerate the insertion of new vinyls into the online store’s catalog. I was asked to design a technical solution capable of automatically retrieving missing data on the internet and formatting it to inject it directly into the database.

My Achievements

Data Extraction Automation (Scraping)

The initial process required an employee to manually search for the artist, copy the tracklist, download covers and MP3 audio snippets, which took an average of 10 minutes per record. I developed an automated Web Scraping script. This bot parses music distributors’ web pages, identifies the HTML structure, and extracts metadata in a structured manner. The solution completely eliminated the manual search for information.

E-commerce Database Integration

The raw data then had to be inserted into the store’s CMS. I designed a transformation pipeline that formats the extracted data (string cleaning, formatting image and MP3 URLs) and generates insert queries for the existing database. Integration is now a one-click process, ensuring product listing consistency.

My Personal Contribution

I worked entirely autonomously on this development project. I personally handled the design of the script architecture, writing the scraping code, error management, and the final integration with the store’s database.

Technologies Used

• Python → Main language used to develop the scraping script architecture (likely using libraries like BeautifulSoup or Selenium).
• Relational Database → Final target for inserting formatted product listings.
• Regex (Regular Expressions) → Used to parse and clean complex textual metadata from raw HTML.

Technical Challenges

HTML Source Instability

The major challenge of Web Scraping is the volatility of target sites: the structure of web pages changes frequently, and information (like an audio snippet) can sometimes be missing. The problem came from exceptions raised by the script that blocked the total import of a batch of records as soon as information was missing. I solved this problem by implementing robust error management (try/catch system and fallbacks). If data is missing, the script logs the warning but continues importing the rest of the information, making the pipeline resilient.

Results

• Over 500 vinyls were automatically and cleanly imported into the catalog during my 3-month mission.
• Massive time savings: Several hundred hours of manual work were saved, allowing staff to refocus on customer service.

What I Learned

Technical Skills: I gained solid experience in manipulating unstructured data (HTML) and creating resilient scripts (error management at scale). Professional Skills: This experience made me realize the direct impact of code on business processes. I integrated a strong principle: automate repetitive tasks as much as possible to generate business value.