用深度神经网络搭建马赛克神器，高清无码效果感人

=目录

1、项目背景

2、适用范围

3、使用方法

1、项目背景

相信一提起马赛克这个东西，不少小伙伴都痛心疾首，虽然最近几年也频繁传出有在研发去除马赛克的软件，一直没有成品问世。不过最近一位程序员及经过不断努力终于完成了这款软件。

据悉这位程序员“deeppomf”用深度神经网络开发出了一个能抹去马赛克让原图重现的神奇程序：DeepCreamPy 。为了使这款软件达到更好的效果，作者在短短几个月内收集了超过10万张未打码的原图，但其中95%的图片他都没有仔细看过，只因为太过于浪费时间了。软件被上传分享后，在一周内被下载了500多次。不过目前该软件的局限性还很大，只能完成一些简单的修复。





















该项目使用深度完全卷积神经网络（deep fully convolutional neural network），参照了英伟达在今年4月前发布的一篇论文。当然，英伟达原文的目的可不是用来做羞羞的事情，而是为了复原画面被单色条带遮挡的问题。

从实际效果来看，复原后的图片涂抹痕迹仍然比较明显，不过处理线条比较简单的漫画可以说是绰绰有余。 

2、适用范围

DeepCreamPy仅适用于薄码，如果马赛克太大太厚，去码可能会失效。另外，它对真人图片无效。如果你非要尝试，可以看一下强行使用的效果：

而且DeepCreamPy目前的版本还不能完全自动处理图片，需要用Photoshop首先对马赛克部分进行手动预处理。 

3、使用方法

第一步：安装程序

1、如果你是64位Windows用户，恭喜你可以直接下载exe程序

下载地址：https://github.com/deeppomf/DeepCreamPy/releases/latest

2、否则需要自己编译，编译代码需要一下组件：

  • Python 3.6
  • TensorFlow 1.10
  • Keras 2.2.4
  • Pillow
  • h5py

请注意软件版本，Windows上的TensorFlow不兼容Python 2，也不兼容Python 3.7。

代码如下：

  1. import numpy as np
  2. from PIL import Image
  3. import os
  4.  
  5. from copy import deepcopy
  6.  
  7. import config
  8. from libs.pconv_hybrid_model import PConvUnet
  9. from libs.utils import *
  10.  
  11. class Decensor:
  12.  
  13.     def __init__(self):
  14.         self.args = config.get_args()
  15.         self.is_mosaic = self.args.is_mosaic
  16.  
  17.         self.mask_color = [self.args.mask_color_red/255.0, self.args.
  18. mask_color_green/255.0, self.args.mask_color_blue/255.0]
  19.  
  20.         if not os.path.exists(self.args.decensor_output_path):
  21.             os.makedirs(self.args.decensor_output_path)
  22.  
  23.         self.load_model()
  24.  
  25.     def get_mask(self, colored):
  26.         mask = np.ones(colored.shape, np.uint8)
  27.         i, j = np.where(np.all(colored[0] == self.mask_color, axis=-1))
  28.         mask[0, i, j] = 0
  29.         return mask
  30.  
  31.     def load_model(self):
  32.         self.model = PConvUnet()
  33.         self.model.load(
  34.             r"./models/model.h5",
  35.             train_bn=False,
  36.             lr=0.00005
  37.         )
  38.  
  39.     def decensor_all_images_in_folder(self):
  40.         #load model once at beginning and reuse same model
  41.         #self.load_model()
  42.         color_dir = self.args.decensor_input_path
  43.         file_names = os.listdir(color_dir)
  44.  
  45.         #convert all images into np arrays and put them in a list
  46.         for file_name in file_names:
  47.             color_file_path = os.path.join(color_dir, file_name)
  48.             color_bn, color_ext = os.path.splitext(file_name)
  49.             if os.path.isfile(color_file_path) and color_ext.casefold() == ".png":
  50.                 print("-----------------------------------------------------------
  51. ---------------")
  52.                 print("Decensoring the image {}".format(color_file_path))
  53.                 colored_img = Image.open(color_file_path)
  54.                 #if we are doing a mosaic decensor
  55.                 if self.is_mosaic:
  56.                     #get the original file that hasn't been colored
  57.                     ori_dir = self.args.decensor_input_original_path
  58.                     #since the original image might not be a png, test multiple file formats
  59.                     valid_formats = {".png", ".jpg", ".jpeg"}
  60.                     for test_file_name in os.listdir(ori_dir):
  61.                         test_bn, test_ext = os.path.splitext(test_file_name)
  62.                         if (test_bn == color_bn) and (test_ext.casefold() in valid_formats):
  63.                             ori_file_path = os.path.join(ori_dir, test_file_name)
  64.                             ori_img = Image.open(ori_file_path)
  65.                             # colored_img.show()
  66.                             self.decensor_image(ori_img, colored_img, file_name)
  67.                             break
  68.                     else: #for...else, i.e if the loop finished without encountering break
  69.                         print("Corresponding original, uncolored image not found in {}.
  70. ".format(ori_file_path))
  71.                         print("Check if it exists and is in the PNG or JPG format.")
  72.                 else:
  73.                     self.decensor_image(colored_img, colored_img, file_name)
  74.         print("--------------------------------------------------------------------------")
  75.  
  76.     #decensors one image at a time
  77.     #TODO: decensor all cropped parts of the same image in a batch (then i need 
  78. input for colored an array of those images and make additional changes)
  79.     def decensor_image(self, ori, colored, file_name):
  80.         width, height = ori.size
  81.         #save the alpha channel if the image has an alpha channel
  82.         has_alpha = False
  83.         if (ori.mode == "RGBA"):
  84.             has_alpha = True
  85.             alpha_channel = np.asarray(ori)[:,:,3]
  86.             alpha_channel = np.expand_dims(alpha_channel, axis =-1)
  87.             ori = ori.convert('RGB')
  88.  
  89.         ori_array = image_to_array(ori)
  90.         ori_array = np.expand_dims(ori_array, axis = 0)
  91.  
  92.         if self.is_mosaic:
  93.             #if mosaic decensor, mask is empty
  94.             # mask = np.ones(ori_array.shape, np.uint8)
  95.             # print(mask.shape)
  96.             colored = colored.convert('RGB')
  97.             color_array = image_to_array(colored)
  98.             color_array = np.expand_dims(color_array, axis = 0)
  99.             mask = self.get_mask(color_array)
  100.             # mask_reshaped = mask[0,:,:,:] * 255.0
  101.             # mask_img = Image.fromarray(mask_reshaped.astype('uint8'))
  102.             # mask_img.show()
  103.  
  104.         else:
  105.             mask = self.get_mask(ori_array)
  106.  
  107.         #colored image is only used for finding the regions
  108.         regions = find_regions(colored.convert('RGB'))
  109.         print("Found {region_count} censored regions in this image!".format
  110. (region_count = len(regions)))
  111.  
  112.         if len(regions) == 0 and not self.is_mosaic:
  113.             print("No green regions detected!")
  114.             return
  115.  
  116.         output_img_array = ori_array[0].copy()
  117.  
  118.         for region_counter, region in enumerate(regions, 1):
  119.             bounding_box = expand_bounding(ori, region)
  120.             crop_img = ori.crop(bounding_box)
  121.             # crop_img.show()
  122.             #convert mask back to image
  123.             mask_reshaped = mask[0,:,:,:] * 255.0
  124.             mask_img = Image.fromarray(mask_reshaped.astype('uint8'))
  125.             #resize the cropped images
  126.             crop_img = crop_img.resize((512, 512))
  127.             crop_img_array = image_to_array(crop_img)
  128.             crop_img_array = np.expand_dims(crop_img_array, axis = 0)
  129.             #resize the mask images
  130.             mask_img = mask_img.crop(bounding_box)
  131.             mask_img = mask_img.resize((512, 512))
  132.             # mask_img.show()
  133.             #convert mask_img back to array 
  134.             mask_array = image_to_array(mask_img)
  135.             #the mask has been upscaled so there will be values not equal to 0 or 1
  136.  
  137.             mask_array[mask_array > 0] = 1
  138.             
  139.             if self.is_mosaic:
  140.                 a, b = np.where(np.all(mask_array == 0, axis = -1))
  141.                 print(a, b)
  142.                 coords = [coord for coord in zip(a,b) if ((coord[0] + coord[1]) % 2 
  143. == 0)]
  144.                 a,b = zip(*coords)
  145.  
  146.                 mask_array[a,b] = 1
  147.                 # mask_array = mask_array * 255.0
  148.                 # img = Image.fromarray(mask_array.astype('uint8'))
  149.                 # img.show()
  150.                 # return
  151.  
  152.             mask_array = np.expand_dims(mask_array, axis = 0)
  153.  
  154.             # Run predictions for this batch of images
  155.             pred_img_array = self.model.predict([crop_img_array, mask_array, mask_array])
  156.             
  157.             pred_img_array = pred_img_array * 255.0
  158.             pred_img_array = np.squeeze(pred_img_array, axis = 0)
  159.  
  160.             #scale prediction image back to original size
  161.             bounding_width = bounding_box[2]-bounding_box[0]
  162.             bounding_height = bounding_box[3]-bounding_box[1]
  163.             #convert np array to image
  164.  
  165.             # print(bounding_width,bounding_height)
  166.             # print(pred_img_array.shape)
  167.  
  168.             pred_img = Image.fromarray(pred_img_array.astype('uint8'))
  169.             # pred_img.show()
  170.             pred_img = pred_img.resize((bounding_width, bounding_height), 
  171. resample = Image.BICUBIC)
  172.  
  173.             pred_img_array = image_to_array(pred_img)
  174.  
  175.             # print(pred_img_array.shape)
  176.             pred_img_array = np.expand_dims(pred_img_array, axis = 0)
  177.  
  178.             # copy the decensored regions into the output image
  179.             for i in range(len(ori_array)):
  180.                 for col in range(bounding_width):
  181.                     for row in range(bounding_height):
  182.                         bounding_width_index = col + bounding_box[0]
  183.                         bounding_height_index = row + bounding_box[1]
  184.                         if (bounding_width_index, bounding_height_index) in region:
  185.                             output_img_array[bounding_height_index][bounding_
  186. width_index] = pred_img_array[i,:,:,:][row][col]
  187.             print("{region_counter} out of {region_count} regions decensored."
  188. .format(region_counter=region_counter, region_count=len(regions)))
  189.  
  190.         output_img_array = output_img_array * 255.0
  191.  
  192.         #restore the alpha channel if the image had one
  193.         if has_alpha:
  194.             output_img_array = np.concatenate((output_img_array, alpha_channel), 
  195. axis = 2)
  196.  
  197.         output_img = Image.fromarray(output_img_array.astype('uint8'))
  198.  
  199.         #save the decensored image
  200.         #file_name, _ = os.path.splitext(file_name)
  201.         save_path = os.path.join(self.args.decensor_output_path, file_name)
  202.         output_img.save(save_path)
  203.  
  204.         print("Decensored image saved to {save_path}!".format(save_path=save_path))
  205.         return
  206.  
  207. if __name__ == '__main__':
  208.     decensor = Decensor()
  209.     decensor.decensor_all_images_in_folder()

注意：运行Demo需要翻墙下载模型，这里为了方便小伙伴，我已经下载完毕：https://download.csdn.net/download/m0_38106923/10798221

第二步：手动处理黑条遮挡和马赛克遮挡

首先打开Photoshop或者其它图片处理器。对于单色条遮住人物敏感部位的情况，使用纯绿色(色号#00FF00#)预处理图片，以绿条取代图片中的黑条。

强烈建议使用铅笔而不是毛刷工具，如果使用毛刷，请一定要关闭抗锯齿功能。或者用魔棒选中马赛克区域，再用油漆桶上色。这里我给各位分享一个Python编写的处理工具，代码比较长，所以和模型文件放在一起，各位可以尝试下载。

最后将处理的文件以PNG格式存储在软件的”decensor_input”文件夹中。如果敏感部位不是黑条，而是马赛克，还需要将未上色的原始图片放入”decensor_input_original” 文件夹中，并确保其文件名和放在”decensor_input”中的预处理图片文件名相同

第三步：运行去马赛克软件

1、64位Windows用户下载程序的exe后，双击软件即可

2、自行编译项目的用户，需要执行以下两个命令

对于黑条遮挡的图片，输入以下命令：

$ pythondecensor.py

对于马赛克遮挡的图片，输入以下命令：

$ python decensor.py —is_mosaic=True

注意事项

如果你图片处理后成了这样：























一定是你处理的姿势不对，请注意不要犯以下两种错误：

第一幅图中，图片马赛克区域没有完全涂满；第二幅图中，由于开启了抗锯齿功能，导致马赛克边缘区域不是纯绿色填充，请关闭抗锯齿功能！！ 


关注公众号，发送关键字：Java车牌识别，获取项目源码。