Internship Experience at CFIA

Training and Technical Skills:

• Learning Docker, Azure, and the OpenAI API.
• Using PyTorch to create AI models.
• Enhancing Python skills.
• Mastering issue and PR management on GitHub.

Projects Completed:

1. HTML Content Extraction to JSON:

   Objective: Introduce a function that extracts the content of all seeds in the HTML content and converts it to a usable JSON format.

   • Retrieve all HTML pages related to the seeds.
   • Establish a connection with OpenAI's Azure Chat.
   • Ask ChatGPT to process the files and provide a JSON output.
   • Transform the JSON data into a format suitable for the database.
   • Establish a normalized database structure to store this data.

   GitHub Links: Data Transformation





2. Content Analysis and Rating System for Quality Control Using OpenAI and Azure:

   Problem Description: Create a comprehensive system that extracts URLs from a piece of web page, uses the OpenAI API via Azure to generate relevant questions and answers, and uses a search function to rate the accuracy of the top ten search results against the generated questions and answers.

   1. URL Extraction: Extract the piece using a query on the database.
   2. Question-Answer Generation: Use the Azure OpenAI API to generate questions and corresponding answers based on the content.
   3. Piece Extraction: Implement a method to extract text pieces from the collected data.
   4. Generated Question-Answer Set: Create a dataset of Question-Answer pairs from the conversational agent's responses.

   GitHub Links: Question Generation

I also worked in Open Source and managed and restructured schemas and containers related to the database.