caffe:how to concat two blobs after ImageData Layers - concatenation

Question

I am trying to detect changed pixels from two images which are took in different time at the same place, so I used slice window to get 5x5 patches for every pixel in two images.
After that, I got 45000 patches for each image, and I selected 35000 patches randomly for training and the others for testing (two images use the same selection).
Then I stored in train/train1 and train/train2, val/val1 and val/val2.
The train1.txt looks like path/to/train/train1/1_00001.png 0,train2.txt looks like path/to/train/train2/2_00001.png 0,
1_00001 and 2_00001 are corresponding pixels in two images, and label=0 means unchanged, label=1 means changed, val1.txt and val2.txt are similar.
Instead of converting into lmdb or leveldb, I used ImageData Layer directly, after DataLayer,I aimed to concat two blobs in channel(the two images are single channel), so my train_val.prototxt is:
name: "ChangeDetec"
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
scale: 0.00390625
}
image_data_param {
source: "data/changedetec/train1.txt"
batch_size: 500
}
}
layer {
name: "pair_data"
type: "ImageData"
top: "pair_data"
include {
phase: TRAIN
}
transform_param {
scale: 0.00390625
}
image_data_param {
source: "data/changedetec/train2.txt"
batch_size: 500
}
}
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
include {
phase: TEST
}
transform_param {
scale: 0.00390625
}
image_data_param {
source: "data/changedetec/val1.txt"
batch_size: 1000
}
}
layer {
name: "pair_data"
type: "ImageData"
top: "pair_data"
include {
phase: TEST
}
transform_param {
scale: 0.00390625
}
image_data_param {
source: "data/changedetec/val2.txt"
batch_size: 1000
}
}
layer {
name: "channelconcat"
type: "Concat"
bottom: "data"
bottom: "para_data"
top: "concatdata"
}
layer {
name: "conv1/5x5"
type: "Convolution"
bottom: "concatdata"
top: "conv1/5x5"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
pad: 0
kernel_size: 5
stride: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0.2
}
}
}
layer {
name: "conv1/relu_5x5"
type: "ReLU"
bottom: "conv1/5x5"
top: "conv1/5x5"
}
layer {
name: "conv2/1x1"
type: "Convolution"
bottom: "conv1/5x5"
top: "conv2/1x1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 64
kernel_size: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0.2
}
}
}
layer {
name: "conv2/relu_1x1"
type: "ReLU"
bottom: "conv2/1x1"
top: "conv2/1x1"
}
layer {
name: "conv3/1x1"
type: "Convolution"
bottom: "conv2/1x1"
top: "conv3/1x1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 32
pad: 0
kernel_size: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0.2
}
}
}
layer {
name: "conv3/relu"
type: "ReLU"
bottom: "conv3/1x1"
top: "conv3/1x1"
}
layer {
name: "conv4/1x1"
type: "Convolution"
bottom: "conv3/1x1"
top: "conv4/1x1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 16
pad: 0
kernel_size: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0.2
}
}
}
layer {
name: "conv4/relu"
type: "ReLU"
bottom: "conv4/1x1"
top: "conv4/1x1"
}
layer {
name: "conv5/1x1"
type: "Convolution"
bottom: "conv4/1x1"
top: "conv5/1x1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 8
pad: 0
kernel_size: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0.2
}
}
}
layer {
name: "conv5/relu"
type: "ReLU"
bottom: "conv5/1x1"
top: "conv5/1x1"
}
layer {
name: "conv6/1x1"
type: "Convolution"
bottom: "conv5/1x1"
top: "conv6/1x1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 4
pad: 0
kernel_size: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0.2
}
}
}
layer {
name: "conv6/relu"
type: "ReLU"
bottom: "conv6/1x1"
top: "conv6/1x1"
}
layer {
name: "conv7/1x1"
type: "Convolution"
bottom: "conv6/1x1"
top: "conv7/1x1"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
convolution_param {
num_output: 2
pad: 0
kernel_size: 1
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
value: 0.2
}
}
}
layer {
name: "conv7/relu"
type: "ReLU"
bottom: "conv7/1x1"
top: "conv7/1x1"
}
layers {
name: "pool"
bottom: "conv7/1x1"
top: "pool"
type: "Pooling"
pooling_param {
pool: AVE
kernel_size: 1
stride: 1
}
}
layer {
name: "accuracy"
type: "Accuracy"
bottom: "pool"
bottom: "label"
top: "accuracy"
include {
phase: TEST
}
}
layers {
name: "loss"
type: SoftmaxWithLoss
bottom: "pool"
bottom: "label"
top: "loss"
include: { phase: TRAIN }
}
and solver.prototxt is:
net: "examples/changedetec/train_val.prototxt"
test_iter: 500
test_interval: 500
base_lr: 0.001
lr_policy: "step"
gamma: 0.1
stepsize: 100000
display: 20
max_iter: 450000
momentum: 0.9
weight_decay: 0.0005
snapshot: 5000
snapshot_prefix: "examples/changedetec/changedetec_train"
solver_mode: GPU
But I got the error:
I0812 10:33:31.980985 3846 solver.cpp:91] Creating training net from net file: examples/changedetec/train_val.prototxt
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 11:11: Expected double.
F0812 10:33:31.981097 3846 upgrade_proto.cpp:79] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: examples/changedetec/train_val.prototxt
*** Check failure stack trace: ***
# 0x7f148e770daa (unknown)
# 0x7f148e770ce4 (unknown)
# 0x7f148e7706e6 (unknown)
# 0x7f148e773687 (unknown)
# 0x7f148eecae3e caffe::ReadNetParamsFromTextFileOrDie()
# 0x7f148eedf17b caffe::Solver<>::InitTrainNet()
# 0x7f148eee024c caffe::Solver<>::Init()
# 0x7f148eee057a caffe::Solver<>::Solver()
# 0x7f148eea2043 caffe::Creator_SGDSolver<>()
# 0x40ed1e caffe::SolverRegistry<>::CreateSolver()
# 0x407ec2 train()
# 0x405cbc main
# 0x7f148d2cdf45 (unknown)
# 0x40648d (unknown)
# (nil) (unknown)
Aborted (core dumped)
What's wrong in my prototxts or is there something I did wrong?Has there a better way to tackle this problem?This has bothering me so many days,please help me.
update in Aug23
I have fixed some low-level errors and still get some other errors:
I commented scale:0.00390625 because the error says Error parsing text-format caffe.NetParameter: 11:11: Expected double..
Then the original error has been fixed,but the error jumped to Pooling layer,sayed Expected integer or identifier,so I commented Pooling layerand changed the last Convolution layer and ReLU layer's num_output to 2 when I use SoftmaxWithLoss layer,but the error is the same as Pooling layerexcept it happened in SoftmaxWithLoss layer.
I tried SigmoidCrossEntropyLoss with the last Convolution layer and ReLU layer's num_output is 1 but it makes no difference to the matter .
I also convert images into LMDB，but error just as well as 1-3 above.