Landoulsi Mohamed Fares

Contributed to developing a system to generate code in various programming languages and frameworks from user stories, enhancing developer productivity. Responsibilities included:

• Extracting relevant information from user stories using both classical NLP techniques like part-of-speech tagging and modern methods, including LLama 3, to generate descriptive text.
• Analyzing user stories using SroBERTa to ensure they provided sufficient information for processing by GPT-4, enabling accurate code generation.
• Improving prompts passed to LLMs, refining inputs to achieve more precise outputs.
• Generating code from Figma designs by exporting frames and translating navigation behavior between frames into code using the Figma REST API.
• Assisting in deploying LLama 3 on an inference server (Ollama) to ensure scalable and efficient usage.

Experience with LLMs:

• Closed-Source LLMs: Claude 3.5 Sonnet
• Open-Source LLMs: LLama 3.0 (70B) and LLama 3.1 (405B)

This project is a subproject of a smart glasses project that aims to provide assistance for individuals with visual impairment. The whole project contains these subprojects:

• Police gestures recognition, hand gestures recognition, traffic signs recognition, face recognition, obstacle detection.

I was in charge of the obstacle detection system with two of my colleagues:

• This part of the smart glasses project helps visually impaired individuals be fully autonomous in homes and outside by providing an interactive speech recognition module and wake word detection technology. The individual can ask about the environment around him and get accurate descriptions (image captioning model named BLIP) and ask questions about the scene (visual question answering model named ViLT) and search for specific objects (GroundingDino model). Our obstacle detection system detects obstacles using Yolov8, an instance segmentation model.
• I was in charge as well of integrating this project with an Intel RealSense camera model 435i to be able to get the depth estimation and extract distance from the camera to the objects to improve our obstacle detection system.
• I took the responsibility of integrating the system as a whole on the Nvidia Jetson Orin card, including the other parts of the smart glasses project.

Was in charge of developing a chatbot development platform based on Rasa Open source framework to automate the development of chatbots for both chatbot developers and non-tech users.

Have learnt how to use NLP, NLU and REST API's in a real development project and expanded my knowledge on MongoDB, Flask, Javascript, Scrum and UML.