Improved performance of deep learning neural network models for Traffic sign classification using brightness augmentation (99.1% solution).
In a previous post we presented a neural network based solution to obtain 98.8% accuracy on German Traffic Sign Dataset. German traffic sign data set is a benchmark data sets computer vision and machine learning problems. The data set is compiled by the Real-Time Computer Vision group at the Institut für Neuroinformatik, and includes 43 classes of German traffic signs. Table below shows how our model compares with other benchmarks and human accuracy.
To achieve the accuracy of 98.8%, we added random rotations, shearing and translations to the traffic sign image. In that solution, we accounted for changes in lighting conditions by applying a brightness histogram normalization. Another way to make the neural network insensitive to brightness changes is to randomly augment brightness of the training image, so model learns to not rely on brightness information. Recently some one (Kosuke Murakami) asked if there is a significant difference between using brightness augmentation vs histogram normalization, and I was not sure. So I added brightness augmentation after performing histogram equalization. Surprisingly, the model learned the data better, and the accuracy on the set improved from 98.8% to 99.1%.
The model and all the hyper parameters are the same as before. The model consists of first a simple color transformer block applied via (1X1X3) convolution, three convolution blocks followed by a 2 layer fully connected layer.
Only change in our model is the addition of brightness augmentation before preprocessing the input image. Brightness augmentation is implemented via a function shown below.
def augment_brightness_camera_images(image):
image1 = cv2.cvtColor(image,cv2.COLOR_RGB2HSV)
image1 = np.array(image1, dtype = np.float64)
random_bright = .5+np.random.uniform()
image1[:,:,2] = image1[:,:,2]*random_bright
image1[:,:,2][image1[:,:,2]>255] = 255
image1 = np.array(image1, dtype = np.uint8)
image1 = cv2.cvtColor(image1,cv2.COLOR_HSV2RGB)
return image1The brightness augmentation is applied just before preprocessing, as shown below.
def transform_image(image,ang_range,shear_range,trans_range):# Rotationang_rot = np.random.uniform(ang_range)-ang_range/2
rows,cols,ch = image.shape
Rot_M = cv2.getRotationMatrix2D((cols/2,rows/2),ang_rot,1)# Translation
tr_x = trans_range*np.random.uniform()-trans_range/2
tr_y = trans_range*np.random.uniform()-trans_range/2
Trans_M = np.float32([[1,0,tr_x],[0,1,tr_y]])# Shear
pts1 = np.float32([[5,5],[20,5],[5,20]])pt1 = 5+shear_range*np.random.uniform()-shear_range/2
pt2 = 20+shear_range*np.random.uniform()-shear_range/2pts2 = np.float32([[pt1,5],[pt2,pt1],[5,pt2]])shear_M = cv2.getAffineTransform(pts1,pts2)
image = cv2.warpAffine(image,Rot_M,(cols,rows))
image = cv2.warpAffine(image,Trans_M,(cols,rows))
image = cv2.warpAffine(image,shear_M,(cols,rows))
#Brightness augmentation
image = augment_brightness_camera_images(image)
# Preprocessing
image = pre_process_image(image)
return image
Figures below show the result applying brightness augmentation in addition to rotation, augmentation and perspective transformations.
Effect of applying brightness augmentation in addition to rotation, scaling and translation augmentation is shown below.
Reflection:
Its surprising how a small step of adding brightness augmentation improved the performance of our deep learning model. I am curious about how much the accuracy can be improved based on adding such simple transformations. In the coming weeks, I will apply more augmentation such as occlusions, adding random noise to channels, posting image on random backgrounds, to see how much improvement I can get. These results were obtained after 4 hours of training. I haven't performed any hyperparameter (learning rate and regularization) tuning or ran the model for more than 4 hours. I think these results could further be improved by using hyperparameter tuning and training for longer duration. If any of the readers try this and get better performance, please let me know.
