接下来(iter())是在python中使用tensorflow创建数据集时抛出错误
提问于 2020-05-01 15:12:56
我正在尝试运行以下代码行，以使用python中的tensorflow创建dataset。我使用的是tensorflow版本'2.2.0-rc3‘。
data = [[[2107, 1037, 3376, 2154, 1012, 1012, 1012, 10166], 1],
[[3819, 2305, 2000, 2022, 2012, 1996, 3608, 2380], 1]]
all_dataset = tf.data.Dataset.from_generator(lambda: data,
output_types = (tf.int32 , tf.int32))
next(iter(all_dataset))它给出了以下错误：
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/context.py in execution_mode(mode)
1985 ctx.executor = executor_new
-> 1986 yield
1987 finally:
10 frames
InvalidArgumentError: TypeError: `generator` yielded an element that did not match the expected structure. The expected structure was (tf.int32, tf.int32), but the yielded element was [[2107, 1037, 3376, 2154, 1012, 1012, 1012, 10166], 1].
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 791, in generator_py_func
flattened_values = nest.flatten_up_to(output_types, values)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py", line 396, in flatten_up_to
assert_shallow_structure(shallow_tree, input_tree)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py", line 299, in assert_shallow_structure
"Input has type: %s." % type(input_tree))
TypeError: If shallow structure is a sequence, input must also be a sequence. Input has type: <class 'list'>.在处理上述异常的过程中，发生了另一个异常：
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 243, in __call__
ret = func(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py", line 309, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 796, in generator_py_func
"element was %s." % (output_types, values)), sys.exc_info()[2])
File "/usr/local/lib/python3.6/dist-packages/six.py", line 692, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 791, in generator_py_func
flattened_values = nest.flatten_up_to(output_types, values)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py", line 396, in flatten_up_to
assert_shallow_structure(shallow_tree, input_tree)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py", line 299, in assert_shallow_structure
"Input has type: %s." % type(input_tree))
TypeError: `generator` yielded an element that did not match the expected structure. The expected structure was (tf.int32, tf.int32), but the yielded element was [[2107, 1037, 3376, 2154, 1012, 1012, 1012, 10166], 1].
[[{{node PyFunc}}]] [Op:IteratorGetNext]在处理上述异常的过程中，发生了另一个异常：
InvalidArgumentError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/executor.py in wait(self)
65 def wait(self):
66 """Waits for ops dispatched in this executor to finish."""
---> 67 pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
68
69 def clear_error(self):
InvalidArgumentError: TypeError: `generator` yielded an element that did not match the expected structure. The expected structure was (tf.int32, tf.int32), but the yielded element was [[2107, 1037, 3376, 2154, 1012, 1012, 1012, 10166], 1].
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 791, in generator_py_func
flattened_values = nest.flatten_up_to(output_types, values)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py", line 396, in flatten_up_to
assert_shallow_structure(shallow_tree, input_tree)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py", line 299, in assert_shallow_structure
"Input has type: %s." % type(input_tree))
TypeError: If shallow structure is a sequence, input must also be a sequence. Input has type: <class 'list'>.在处理上述异常的过程中，发生了另一个异常：
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 243, in __call__
ret = func(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py", line 309, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 796, in generator_py_func
"element was %s." % (output_types, values)), sys.exc_info()[2])
File "/usr/local/lib/python3.6/dist-packages/six.py", line 692, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 791, in generator_py_func
flattened_values = nest.flatten_up_to(output_types, values)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py", line 396, in flatten_up_to
assert_shallow_structure(shallow_tree, input_tree)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/util/nest.py", line 299, in assert_shallow_structure
"Input has type: %s." % type(input_tree))
TypeError: `generator` yielded an element that did not match the expected structure. The expected structure was (tf.int32, tf.int32), but the yielded element was [[2107, 1037, 3376, 2154, 1012, 1012, 1012, 10166], 1].
[[{{node PyFunc}}]]
回答 1
Stack Overflow用户
发布于 2020-05-01 15:29:02
您说过您的目标是在python中使用tensorflow创建数据集，然后检查一下这是否适用于您，而不是使用from_generator为什么不使用from_tensor_slices
data = [[[2107, 1037, 3376, 2154, 1012, 1012, 1012, 10166], 1],
[[3819, 2305, 2000, 2022, 2012, 1996, 3608, 2380], 1]]
x = np.array([data[i][0] for i in range(len(data))])
y = np.array([data[i][1] for i in range(len(data))])
all_dataset = tf.data.Dataset.from_tensor_slices((x, y))
next(iter(all_dataset))页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/61545217
复制
腾讯云开发者
