Post 1
Published: February 23, 2024
Our initial idea to solve the problem was to create a pole that alerted people (whether they needed help or not) if a moving vehicle was close to the crosswalk that the pedestrian was going to cross and through an audible signal.
However, this idea proved to be too impractical and too complex to solve our challenge. We're excited to continue this journey and bring our vision to life, helping visually impaired individuals gain more independence and safety in their daily lives.
Therefore, following the advice of our advisor Marcelino Santos, we decided to change vehicle detection to two cameras that transmit videos to the cell phone, where image recognition is carried out to detect possible dangers for pedestrians, alerting them with a signal.
Post 2
Published: March 1, 2024
We contacted ACAPO (Association of the Blind and Visually Impaired of Portugal) to assess the feasibility and interest of our idea for a blind individual. In this way, we obtained a positive opinion as well as acquired more information regarding the serious difficulties that a blind person experiences whenever they want to cross the road.
Post 3
Published: March 8, 2024
Right now, we're focused on finding the right hardware to make this device work well. We started to research about efficient cameras, microcontrollers and batteries to power it up.These components must be efficient, compact, and capable of operating under various conditions. Finding the right hardware is essential to making our car detector effective and user-friendly.
Post 4
Published: March 15, 2024
We contacted Professor João Paulo Costeira with the intention of giving us feedback on our idea and advising us on image recognition software.
We found a model of the esp32 that can have a camera attached on its back that can transmit the recording to a webserver
Post 5
Published: March 22, 2024
We met with Professor João Paulo Costeira in order to deepen our knowledge of image processing.
The meeting was very productive, as we were shown several tools to assist in the development of our image recognition software, as well as products to facilitate hardware design.
It also helped us to expand our horizons and consider possible improvements to make to our design, such as the use of a gimbal system to guarantee the stability of the camera.
-We decided to use a 5V powerbank as a power source for the microcontroller.
-We send a request for the material necessary to assemble the camera circuit.
Post 6
Published: March 29, 2024
Due to the limitations of the Bluetooth, we decided to change the comunication system between the camera and the app to a wi-fi based one, specifically a wi-fi created with the user's mobile data (hotspot).
Post 7
Published: April 12, 2024
Due to the exams and final projects, our progress slow down this week.
Post 8
Published: April 19, 2024
We started testing the camera and it's transmission where it succesfully created an IP where any device that is connected to that same wi-fi as the camera can see the video transmission (Antonio Neto).
We have also decided to take the teacher's advice and change the battery from a powerbank to a lithium one for both camera.
Additionally, we also started researching and setting up the project folder and the developing tools for the application (João Gouveia).
Finally, we sent emails to the following image recognition companies and associations asking for feedback about our project:
(Francisco Mendonça,José Lopes de Carvalho)
- Waymo
- Bosh Portugal
- Departamento Car Multimedia
- Critical Software
Application design planning and beginning of it's development in Expo (David Gameiro)
Post 9
Published: April 26, 2024
Due to complications in Expo (which is a javascript-based reactive and native language), we decided to switch the language of our project to python, being easier to implement the image recognition operations.
We also solved some hardware problems that with the upload of arduino code to the esp32's cameras, with the help of Professor Pedro Vitor. (All members)
Post 10
Published: May 3, 2024
(João Gouveia, Antonio Neto, Duarte Cardoso, Francisco Mendonça, José Lopes de Carvalho, at Instituto Superior Técnico Taguspark - 30/4/2024 16:00-18:00)
We tried to troublesoot the transmission from the webserver but the video didn't load properly
Later that week we managed to find a library that can solve this issue (lib request) and managed to connect the transmission to the program. We still have to calibrate the measurements but it can succesfully detect and follow cars (keeping its informations, such as speed and -in the future- distances saved) (João Gouveia)
Post 11
Published: May 10, 2024
We started to make the draft of our poster. We also scheduled a reunion with the Professor João Paulo Costeira in order to help us find the best method and equations to deternime the speed and distace of the vehicles, once they are tracked.
(Francisco Mendonça,José Lopes de Carvalho)
We also started making the the base UI of our App (Duarte Cardoso, David Gameiro)
We are trying diferent methods to estimate the speed to see which one is the most adequate for our case. (João Gouveia, Antonio Neto)
Post 12
Published: May 17, 2024
(All members, at Instituto Superior Técnico Taguspark - 17/5/2024 14:00-17:00)
We started the measures and calibration of our transmission to test if the speed was being measured correctly. In that test we determined that, to calculate the speed wof the vehicle we are going to compare the current position with the position from the previous frame when the car crossed a specific height of the frame.
We realized that the resolution that we were fetching from the webserver is not the one that is shown on the opencv window, so we have to be careful with that.
Additionally, we also concluded that depending of the Wi-fi we are using (hotspot or fixed router) the delay that the transmission has will increase.
Lastly, due to some libraries that are crucial for the object tracking and recognition not being supported on android, we can not import or project to the phone.
Post 13
Published: May 24, 2024
(All members, at Instituto Superior Técnico Taguspark - 22/5/2024 14:00-17:00)
We continued the measurements and calibration of our transmission because the speed measurement was malfunctioning. After a lot of tests, we came to the conclusion that our speed detector was working badly, so we decided to approach this situation in a different way.
(All members Zoom Meeting - 24/5/2023 11:30-14:30)
We had a Zoom session with professor João Paulo Costeira and he gave us a better way to solve our problem. Instead of calculating speeds, we started to develop a new program that evaluates the different regions of the frame. With this, we developed a system that evaluates the risk and determines if the car is in a danger zone. He also recomended us a diferent trained model to determine the speed.
Unfortunately, due to complications with the transmission in the newly recomended pre-trained image recognition models (because the esp32 cameras speed are not as stable as phone/computer ones), we decided to make the warning system alert the pedestrian if a car is nearby the crosswalk, even if its not moving, working more like an extra tool to go with the common ones used by blind people (cane and guide dog).
(Duarte Cardoso)
We also solved a factory issue with the battery not conecting to the module (that converts the voltage of the battery from 3.3 V to 5 V) and succesfully repaired the battery, working like it's intended. We expect to have the prototype assembled by next monday.
We continued the development of the pitch desk presentation and poster.
Post 14
Published: May 31, 2024
(All members, at Instituto Superior Técnico Taguspark - 27/5/2024 16:00-18:00)
We made an improvised assembly of our prototype and tested it. We realised that due to each camera creating a webserver, we can't use the opencv to put both transmissions in the same window or computer (due to each transmission having a different data origin, which makes them not compatible to the command that displays the transmission in the same window).
The system is working as intended, giving the command "wait" while a car is in the list of active alerts and only giving the "go" command when that list is empty, and the cooldown is over
We also finished the poster and pitch deck presentation, only missing the information about the final product that we will put once the final assemble is completed next week.
Post 15
Published: June 7, 2024
(All members, at Instituto Superior Técnico Taguspark - 3/6/2024 11:00-16:00)
We finished the assembly of the final version of our prototype, and tested its transmission and alert system. The alert system is working, but due to the transmisson not being stable it ends up having some breaks, but it works.
We also filmed our footage for the demonstration video to showcase the functionality of our prototype.