求助 pyreader构建问题
扑流扑流 发布于2020-07-24 21:21 浏览:309 回复:4
2
收藏
train_program = fluid.default_main_program()
with fluid.program_guard(train_program, startup_prog):
    with fluid.unique_name.guard():
        train_pyreader, graph_vars = \
            create_model(num_rels=data_generate.num_rels, pyreader_name='train_pyreader')

test_program = fluid.default_main_program()
with fluid.program_guard(test_program, startup_prog):
    with fluid.unique_name.guard():
        test_pyreader, graph_vars = \
            create_model(num_rels=data_generate.num_rels, pyreader_name='test_pyreader')

test_program = test_program.clone(for_test=True)

错误如下:

Traceback (most recent call last):
  File "static_graph_new.py", line 400, in 
    create_model(num_rels=data_generate.num_rels, pyreader_name='test_pyreader')
  File "static_graph_new.py", line 195, in create_model
    tokens, segments, sub_heads, sub_tails, sub_head_pos, sub_tail_pos, obj_heads, obj_tails = fluid.layers.read_file(pyreader)
  File "/home/anaconda3/envs/dp/lib/python3.6/site-packages/paddle/fluid/layers/io.py", line 891, in read_file
    for _ in range(len(reader.desc.shapes()))
  File "/home/anaconda3/envs/dp/lib/python3.6/site-packages/paddle/fluid/layers/io.py", line 891, in 
    for _ in range(len(reader.desc.shapes()))
  File "/home/anaconda3/envs/dp/lib/python3.6/site-packages/paddle/fluid/layer_helper_base.py", line 372, in create_variable_for_type_inference
    stop_gradient=stop_gradient)
  File "/home/anaconda3/envs/dp/lib/python3.6/site-packages/paddle/fluid/framework.py", line 2447, in create_var
    var = Variable(block=self, *args, **kwargs)
  File "/home/anaconda3/envs/dp/lib/python3.6/site-packages/paddle/fluid/framework.py", line 934, in __init__
    dtype))
ValueError: Variable read_file_0.tmp_0 has been created before. The previous data type is VarType.INT64; the new data type is VarType.FP32. They are not matched

 这里出现的问题应该和后面具体的数据输入没有关系,不知道问题到底出在哪里

还有就是想问一下这个变量名 read_file_0.tmp_0 能否自己定义,这样是不是可以避免冲突

这里是定义pyreader的代码,看官方文档好像并没有指定变量名的功能

pyreader = fluid.layers.py_reader(
    capacity = 50,
    shapes = [[BATCH_SIZE, 256], [BATCH_SIZE, 256],
            [BATCH_SIZE, 256], [BATCH_SIZE, 256],
            [BATCH_SIZE, 1], [BATCH_SIZE, 1],
            [BATCH_SIZE, 256, 49], [BATCH_SIZE, 256, 49]],
    dtypes = ['int64', 'int64', 'int64', 'int64', 'float64', 'float64', 'float64', 
           'float64'],
    lod_levels=[0, 0, 0, 0, 0, 0, 0, 0],
    name = pyreader_name,
    use_double_buffer = True)

希望得到大家的指点,谢谢!

收藏
点赞
2
个赞
共4条回复 最后由自尊心3回复于2020-08-04 10:45
#2自尊心3回复于2020-07-25 17:40:47

https://paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/data_preparing/static_mode/use_py_reader.html

现在paddle官方推荐用io.DataLoader,可以参照API介绍改一下。

感觉是program定义重复了,都用了default_main_program,和default_startup_program

你试一下

train_prog = fluid.Program()
train_startup = fluid.Program()

 

test_prog = fluid.Program()
test_startup = fluid.Program()

1
#5自尊心3回复于2020-08-04 10:45:34
#4 扑流扑流回复
后来调整了一下您说的这个方法是有效的,现在问题已经解决了,谢谢!

哈哈,不客气

0
#4扑流扑流回复于2020-07-28 16:31:08
#2 自尊心3回复
https://paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/data_preparing/static_mode/use_py_reader.html 现在paddle官方推荐用io.DataLoader,可以参照API介绍改一下。 感觉是program定义重复了,都用了default_main_program,和default_startup_program 你试一下 train_prog = fluid.Program() train_startup = fluid.Program()   test_prog = fluid.Program() test_startup = fluid.Program()
展开

后来调整了一下您说的这个方法是有效的,现在问题已经解决了,谢谢!

0
#3扑流扑流回复于2020-07-27 15:31:42

您说的这个方法我试了一下,问题依然存在

还是改成DataLoader吧

十分感谢回答!

0
TOP
切换版块