My original idea for the Fab academy was to build a scalable vinyl cutter mostly made from easily available materials such as aluminum linear rails and circular pipes that can be sourced from hardware stores. This idea it inspired by many other projects in Fabacademy about scalable fablab tools. In my local fablab Aalto, Our vinyl cutter is the most reliable machine that never gave us a problem and thus the most overlooked. But their is always a size limitations and the price of a large plotter gets expensive.
One of the most repititive and insuficient practices in my lab in giving a tour to visitors to the lab. As a general note we are quit poor in documentation and idea sharing. One solution we had was having a camera that contantly post the project pictures to flicker. As the number of pictures increases the idea gets lost, no way to search more info and contact the maker...
The idea of my final project is thus to have a tour guide. For simplicity, I will now make a shelf displaying the example products we made in the lab. When a visitor picks one product, a tv screen would show all the steps covered to make it. This would require a computer vision setup interated to the shelf.
In Computer Vision field exists several libraries which implements SFM in differents languages as C++, Python, Matlab. So I choose the library OpenFSM which is developed in Python . These framework requires some dependencies as:
#this function display the image gray converted from cam
import cv2
def main():
cam = cv2.VideoCapture(0)
cam.set(3,640)
cam.set(4,480)
while True:
#read image form camera
val, img = cam.read()
if( val == False):
continue
#convert to gray image
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
laplacian = cv2.Laplacian(gray,cv2.CV_64F)
#display the image
cv2.imshow("my_cam", img)
cv2.imshow("laplacian", laplacian)
c = cv2.waitKey(10)
if c == 27: #check the keyboard each 10 ms
break # escape to quit
elif c == ord('s'):
cv2.imwrite('my_cam1.jpg',img)
elif c == ord('t'):
cv2.imwrite('my_cam2.jpg',img)
cam.release()# release the cam
cv2.destroyAllWindows() #destroy the windows
if __name__ == '__main__':
main()
This image got after running the source code:
The other packages OpenGV and OpenSFM are installed according
OpenSFM tutorial
. The example of testing OpenSFM is detailed in the following
steps:
The idea was installing PyQt for python 2.7 because Gestalts nodes work with this version. So I must compile the source with the following steps:
tar -xvf sip-4.18.tar.gz cd /sip-4.18 python configure.py -d /usr/local/lib/python2.7/site-packages/ make make install cd.. tar -xvf PyQt-gpl-5.6.tar.gz cd PyQt-gpl-5.6 python configure.py -d /usr/local/lib/python2.7/site-packages/ --qmake=/usr/local/Cellar/qt5/5.6.0/bin/qmake --sip=/usr/local/bin/sip --sip-incdir=../sip-4.18/siplib make make installTo integrate PyQt with Qt Designer I follow the tutorial . Additionally, I installed Android Studio and apache2 in my Ubuntu 14.04.
The image belong here shows the software architecture for processing image. The images will acquired by mobile phone through an android app which takes a photo in a position given by the machine. The image will transfer to the computer via wifi network. In computer side I run program to process the images. The comunication between phone and computer used UDP socket, the server socket was in computer and the phone was a client.
In this case I used a UDP socket in python, the server side recv datagram from phone through client UPD phone. To achieve the communication line using IP and fixed. The source code for server that uses here.
import socket
import sys
import time
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Bind the socket to the port
server_address = ("172.43.1.11", 44450)
print >>sys.stderr, 'starting up on %s port %s' % server_address
#sock.bind(server_address)
#sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST,1)
while True:
try:
message = 'This is the message. It will be repeated.'
message = raw_input('message: ')
sent = sock.sendto(message, server_address)
time.sleep(3)
except KeyboardInterrupt:
raise
sock.close()
The program did not upload image with more than 2 Mb, so I modified two files: the first one was /etc/apache2/apache2.conf in the line LimitRequesLine 1000000, LimitRequestFieldSize 1000000. Additionaly, I modified /etc/php5/apache2/php.ini the line upload_max_filesize=10M. After restart apache server "sudo service apache2 restart", a firefox plugin "Httprequester" was installed to process the webservice. It can send request, in our case I sent fields with real data as image_name and encoded_string, the last one was compress an image with base64 method. Tutorial of php file uploading .
<?php
// Path to move uploaded files
$target_path = dirname(__FILE__).'/uploads/';
if (isset($_FILES['image']['name'])) {
$target_path = $target_path . basename($_FILES['image']['name']);
try {
// Throws exception incase file is not being moved
if (!move_uploaded_file($_FILES['image']['tmp_name'], $target_path)) {
// make error flag true
echo json_encode(array('status'=>'fail', 'message'=>'could not move file'));
}
// File successfully uploaded
echo json_encode(array('status'=>'success', 'message'=>'File Uploaded'));
} catch (Exception $e) {
// Exception occurred. Make error flag true
echo json_encode(array('status'=>'fail', 'message'=>$e->getMessage()));
}
} else {
// File parameter is missing
echo json_encode(array('status'=>'fail', 'message'=>'Not received any file'));
}
?>
The android app have implemented with library Volley plus following the tutorial to upload the files image to the server. The layout app changed to a new GUI which lunchs the camera, to complete this idea I read post . The camera has own classes to interaction with the app. Now, the challenge was to create a VolleyPlus library to sincronize image captured from the app with GUI program in desktop. Additionaly, this app was listening UDP socket. The app wait the message to start taking picture and upload each one. The source can download in this
I followed the tutorial to create a GUI with python and Qt to control the machine with gestalt nodes, and communicate with android app to take and upload picture. After that it will run OpenSFM framework. It was the idea but in reality I had issues with gestalt code and PyQt because they had different kind of threads. It was impossible to syncronize gestalts and machine, so I use the GUI to send images from phone and manually write the commands to generate 3D model. The GUI source code can download
import sys
from PyQt5.QtWidgets import QApplication, QMainWindow, QTreeView, QFileSystemModel, QTextEdit
from PyQt5.QtCore import QDir, Qt
from .gui.mainwindow_ui import Ui_MainWindow
from plotter import virtualMachine
import socket
import threading
import fcntl
import struct
import time
import os
import math
class MainWindow(QMainWindow, Ui_MainWindow):
def __init__(self):
super(MainWindow, self).__init__()
self.setupUi(self)
self.processingButton.clicked.connect(self.Calculate)
self.sessionButton.clicked.connect(self.Session)
self.picturesButton.clicked.connect(self.Pictures)
self.endButton.clicked.connect(self.End)
self.gestaltButton.clicked.connect(self.gestalt_init)
self.cycle = True #cycle of thread recv_message
def __del__(self):
print "close all"
self.s.close()
self.cycle = False
self.th.stop()
def gestalt_init(self):
##gestaltInterface init function
self.stages = virtualMachine(persistenceFile = "test.vmp")
self.stages.xyNode.setVelocityRequest(4)
self.stages.xNode.setVelocityRequest(4)
self.stages.yNode.setVelocityRequest(8)
print "geslts have just initialized"
gestalt_define_mov()
#function to define the movements of gestalts
def gestalt_define_mov(self):
# Disk
self.dangle= math.ceil(360.0/self.nshots)
self.movi= range(0,361,dangle)
self.movesx = []
for i in self.movi:
self.movesx.append([i])
# Phone
self.arcpoints=[[10],[20],[30],[40],[50],[60],[70],[80],[90],[100]] # Empirical points selected on the arc
self.dangle=360/self.nlevels
self.movi= range(0,361,self.dangle)
self.movesy = []
for i in self.movi:
self.movesy.append([i])
print "moves (x,y) are saving"
##function to start the movements of robot
def gestalt_mov_start(self):
for my in self.movesy: # Phone
self.stages.move(my,0)
status = self.stages.xAxisNode.spinStatusRequest()
# This checks to see if the move is done.
while status['stepsRemaining'] > 0:
time.sleep(0.001)
status = self.stages.xAxisNode.spinStatusRequest()
for mx in self.movesx: # Disk
self.stages.move(mx,0)
status = self.stages.yAxisNode.spinStatusRequest()
# This checks to see if the move is done.
while status['stepsRemaining'] > 0:
time.sleep(0.001)
status = self.stages.yAxisNode.spinStatusRequest()
#here the scoket moments
def gestalt_triggerx(self, mx):
self.stages.move(mx,0)
status = self.stages.xAxisNode.spinStatusRequest()
while status['stepsRemaining'] > 0:
time.sleep(1)
status = self.stages.xAxisNode.spinStatusRequest()
def gestalt_triggery(self, my):
self.stages.move(my,0)
status = self.stages.yAxisNode.spinStatusRequest()
while status['stepsRemaining'] > 0:
time.sleep(1)
status = self.stages.yAxisNode.spinStatusRequest()
def socket_init(self):
self.port = 44450
self.s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self.server_address = (self.client, self.port)
print >>sys.stderr, 'starting up on %s port %s' % self.server_address
self.s.bind(('',self.port))
self.ip = self.get_ip_address('wlan0')
self.ip_line.setText(self.ip)
#self.s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST,1)
def get_ip_address(self, ifname):
return socket.inet_ntoa(fcntl.ioctl( self.s.fileno(),0x8915, # SIOCGIFADDR
struct.pack('256s', ifname[:15]))[20:24])
def send_message(self, text):
self.s.sendto(text, self.server_address)
def recv_message(self):
print >>sys.stderr, 'listen up on %s port %s' % self.server_address
self.sock = self.s
self.th = threading.Thread(target = self.recv_message_thread, args=(self.sock,self.server_address))
self.th.start()
def recv_message_thread(self, client, address):
size = 1024
print "RECV MSG FROM THREAD"
try:
while self.cycle:
data = client.recv(size)
if data:
print "recv from android: " + data
else:
raise error('Client error')
time.sleep(1)
except KeyboardInterrupt:
client.close()
return False
client.close()
def Calculate(self):
self.sessionT = self.sesion_line.text()
self.nshots = self.shots.value()
self.nlevels = self.levels.value()
self.ntime = self.time.value()
self.client = self.ip_server.text()
print self.sessionT
print self.nshots
print self.nlevels
print self.ntime
self.read_items()
self.socket_init()
#self.send_message("SESSION,"+self.sessionT+","+self.ip)
#self.recv_message()
def Session(self):
msg = "SESSION," + self.sessionT+","+self.ip
print 'start session: '+msg
self.send_message(msg)
def Pictures(self):
for x in range(1):
msg = "OK," + self.sessionT+","+self.ip
print 'take Pictures: '+msg
self.send_message(msg)
time.sleep(10)
def End(self):
msg = "END,"+self.sessionT+","+self.ip
print 'start session: '+msg
self.send_message(msg)
model = QFileSystemModel()
model.setRootPath(QDir.currentPath())
self.treeView.setModel(model)
self.treeView.setDragDropMode(QtGui.QAbstractItemView.InternalMove)
def read_items(self):
self.sesion_line.setReadOnly(True)
self.shots.setReadOnly(True)
self.levels.setReadOnly(True)
self.time.setReadOnly(True)
def write_items(self):
self.sesion_line.setReadOnly(False)
self.shots.setReadOnly(False)
self.levels.setReadOnly(False)
self.time.setReadOnly(False)
def main():
app = QApplication(sys.argv)
main_window = MainWindow()
main_window.show()
sys.exit(app.exec_())
if __name__ == "__main__":
main()