SRNet - Editing Text in the Wild
This is a reproducing of paper Editing Text in the wild by tensorflow, which aims to replace or modify a word in the source image with another one while maintaining its realistic look.
Original paper: Editing Text in the wild by Liang Wu, Chengquan Zhang, Jiaming Liu, Junyu Han, Jingtuo Liu, Errui Ding and Xiang Bai.
The model in this project is a result of my experiment and debugging of the details described in the paper.
A pre-trained vgg19 model is used in this SRNet, which is downloaded from https://github.com/fchollet/deep-learning-models/releases/tag/v0.1 and converted to pb format
Data is completely prepared as described in the paper.
You can refer to and improve Synthtext project to render styled texts on background images. Also need to save some intermediate results as labels while rendering.
According to this paper, you need to prepare 2 input images(i_s, i_t) and 4 label images(t_sk, t_t, t_b, t_f)
i_s: styled text a rendering on background image
i_t: standard text b rendering on gray background
t_sk: skeletonization of styled text b.
t_t: styled text b rendering on gray background
t_b: background image
t_f: styled text b rendering on background image
In my experiment, I found it easier to train with one more label data(mask_t).
mask_t: the binary mask of styled text b
From left to right, from top to bottom are examples of
i_s, i_t, t_sk, t_t, t_b, t_f, mask_t
This is our code to generate SRNet data: SRNet-Datagen
Train your own dataset
First clone this project
$ git clone https://github.com/youdao-ai/SRNet.git
Once the data is ready, put the images in different directories with the same name.
You can modify the data directories and training parameters in
cfg.py as you want.
python3 train.py to start training.
This is our pretrained weight: https://pan.baidu.com/s/1TaliSQ96RtlJLSgpOzkRSA, Extraction code: x3qv
You can predict your own data with
$ python3 predict.py --i_s xxx --i_t xxx --save_dir xxx --checkpoint xxx
If you want to predict a directory of data, just make sure your data
i_t have the same prefix and splited by '_', for example,
image1_i_t.png, put them into one directory and
$ python3 predict.py --input_dir xxx --save_dir xxx --checkpoint xxx
Or you can set these path information in
cfg.py and just