student innovation project Brief: ADaptive computer Vision

---

The technical fields that are associated with this innovation are Artificial intelligence using neural networks and robotic motor control. The neural networks are mainly used to identify objects and to save new objects to a memory line for later recollection. The motor control is used so that the robot can move around the area.

This inspiration for this project came from a period of time where I worked up at the corporate head quarters for GameStop.  We were given access to a online classrooms through Udemy. I chose some classes on Neural Networks and how they work. Through this I realized that convolutional neural networks could be used to process and predict images. While also using a recurrent neural network to see what information needs to be saved and what does not. I want to meld these two neural networks into one so that a robot can have complete control over what it retains as useful information while it travels through a new space. 

The prior art for this project is found in data science and analytics.  The convolutional and recurrent neural networks are used independently from each other to find certain data in images or large data sets. For a convolutional network, it takes a series of images and uses feature scaling to give an image a number that it then classifies it. At the end of this neural network it then can make a prediction about an image that is new to the system. In a recurrent neural network, it takes a whole series of data in the form of a cvs file and then can find trends in the large data files and predict how the data will turn later down the road. 

This project aims to meld two neural networks together to form a adaptive learning system for robotics. As the robot looks around with its camera. As it is recording video, the video will be saved in frames so that the neural networks can go to work. Once these frames are saved. They are passed to the convolutional neural network for feature scaling. This process is ultimately designed to look over each cell of an image and assign a number to each cell. After this is done for the whole image. The image itself is given a number and saves that image and number as a combo. After all of this information is gathered it will then be passed to the recurrent neural network. During the recurrent neural network processing. It will take the information from the convolutional neural network and compare it to what is already stored. If it is something new, then it will save it to the memory line. If it is something that is already saved. It will delete the extra information. 

The innovation for this project is coming from the fact that the original neural networks that this project is based around is typically used in data science rather than robotics. The second innovation is that these neural networks are typically used independently of each other. This project aims to meld those two complex neural networks into one so that new images can be saved while repeat images get discarded. 

The usage scenario is quite broad. For any robotic system that uses cameras or any sort of vision hardware can use this innovation.  

The evaluation criteria for this project is how well the system saves the videos as frames. Then how quickly the neural network processes each frame and how many new frames are saved after each video. After that once the images are processed they should pass through both neural networks and save the new frames. 

Objective One: Capturing video 

1. Make sure that the videos are saved.
2. Convert the videos into frames.
3. Pass those frames into the neural network.

Objective Two: Convolutional Neural Network

1. Make sure that it is pulling in the right images from the test set and the frames that are saved. 
2. Make sure that the out put is going to the same place each time.
3. Make sure that the out put data is stored and not automatically lost.

Objective Three: Recurrent Neural Network

1. Make sure that it takes the data from the convolutional neural network.
2. Ensure that the process is working for each data point.
3. Ensure that the proper information is being saved. 

The prototype for this project is a simple raspberry pi with a pi camera connected to it. In the next prototype I will put this raspberry pi on a set of tracks and give the system movement. However, for now I am going to stick with the computer vision aspect. 

The evaluation criteria is to make sure that each piece of data is changed as needed and saved when it needs to be. I will probably even set up the program to save the data so I can evaluate the changes that take place between each step of the neural network process. This will ensure that each set is working as its intended. 

This will be filled out at a later date. 

Our robotics consulting services provide expert guidance and support to help you make informed decisions about your robotics solutions. We work closely with you to understand your business needs and develop customized solutions that meet your specific requirements.