I'm trying to convert a numpy array to a MemoryView object because I have to communicate between two programs. The one can only handle NumPy arrays and the other only MemoryView objects.
Converting from MemoryView to numpy array is easily done by:
import numpy as np
MyNumpyArray=np.array(MyMemoryView)
But how do you convert from numpy array to MemoryView?
I found here: https://docs.python.org/3/c-api/memoryview.html That there's a PyMemoryView_FromObject(PyObject *obj) function, but I don't know how to call it without an example.
Thanks!
memoryview is one of the built-in types and can simply be called as:
arr = np.random.rand(5,4)
view = memoryview(arr)
view
<memory at 0x12699c318>
Additionally to accepted answer providing another simple method to get memoryview out of Numpy array:
Try it online!
a = np.arange(1, 9)
view = a.data
print(type(view)) # <class 'memoryview'>
In other words .data attribute of array variable gives exactly memoryview.
Related
I'm trying to create a few array types for a scientific python project. So far, I have created generic types for 1D, 2D and ND numpy arrays:
from typing import Any, Generic, Protocol, Tuple, TypeVar
import numpy as np
from numpy.typing import _DType, _GenericAlias
Vector = _GenericAlias(np.ndarray, (Tuple[int], _DType))
Matrix = _GenericAlias(np.ndarray, (Tuple[int, int], _DType))
Tensor = _GenericAlias(np.ndarray, (Tuple[int, ...], _DType))
The first issue is that mypy says that Vector, Matrix and Tensor are not valid types (e.g. when I try myvar: Vector[int] = np.array([1, 2, 3]))
The second issue is that I'd like to create a generic type Batch that I'd like to use like so: Batch[Vector[complex]] should be like Matrix[complex], Batch[Matrix[float]] should be like Tensor[float] and Batch[Tensor[int] should be like Tensor[int]. I am not sure what I mean by "should be like" I guess I mean that mypy should not complain.
How to I get about this?
You should not be using protected members (names starting with an underscore) from the outside. They are typically marked this way to indicated implementation details that may change in the future, which is exactly what happened here between versions of numpy. For example in 1.24 your import line from numpy.typing fails at runtime because the members you try to import are no longer there.
There is no need to use internal alias constructors because numpy.ndarray is already generic in terms of the array shape and its dtype. You can construct your own type aliases fairly easily. You just need to ensure you parameterize the dtype correctly. Here is a working example:
from typing import Tuple, TypeVar
import numpy as np
T = TypeVar("T", bound=np.generic, covariant=True)
Vector = np.ndarray[Tuple[int], np.dtype[T]]
Matrix = np.ndarray[Tuple[int, int], np.dtype[T]]
Tensor = np.ndarray[Tuple[int, ...], np.dtype[T]]
Usage:
def f(v: Vector[np.complex64]) -> None:
print(v[0])
def g(m: Matrix[np.float_]) -> None:
print(m[0])
def h(t: Tensor[np.int32]) -> None:
print(t.reshape((1, 4)))
f(np.array([0j+1])) # prints (1+0j)
g(np.array([[3.14, 0.], [1., -1.]])) # prints [3.14 0. ]
h(np.array([[3.14, 0.], [1., -1.]])) # prints [[ 3.14 0. 1. -1. ]]
The issue currently is that shapes have almost no typing support, but work is underway to implement that using the new TypeVarTuple capabilities provided by PEP 646. Until then, there is little practical use in discriminating the types by shape.
The batch issue should be a separate question. Try and ask one question at a time.
I have an empty unicode array:
a = np.array([], dtype=np.str_)
I want to encode it:
b = np.char.encode(a, encoding='utf8')
Why is the result an empty array with dtype=float64?
# array([], dtype=float64)
If the array is not empty the resulting array is a properly encoded array with dtype=|S[n]:
a = np.array(['ss', 'ff☆'], dtype=np.str_)
b = np.char.encode(a, encoding='utf8')
# array([b'ss', b'ff\xe2\x98\x86'], dtype='|S5')
EDIT: The accepted answer below does, in fact, answer the question as posed but if you come here looking for a workaround, here is what I did:
if array.size == 0:
encoded_array = np.chararray((0,))
else:
encoded_array = np.char.encode(a, encoding='utf8')
This will produce an empty encoded array with dtype='|S1' if your decoded array is empty.
The source of numpy.char.encode is available here. It basically calls _vec_string which returns an empty array of type np.object_ in this case. This result is provided to _to_string_or_unicode_array which builds the final array and determine its type. Its code is available here. It basically converts the Numpy array to a list so to then provide it to np.asarray. The goal of this operation is to determine the type of the array but the thing is that empty arrays have a default type of np.float64 by convention (I think it is because Numpy was initially design for physicist who usually work with np.float64 arrays). This result is quite unexpected in this context, but the "S0" does not exists and I am not sure everyone would agree that the "S1" type is better here (still, it is certainly better than np.float64). Feel free to fill an issue on the GitHub Numpy repository so to start a discussion about this behaviour.
I have multiple numpy arrays with different objects. How can I give 'type-hint' to the Pycharm so that I can get methods of that class while coding? For example,
import numpy as np
class Test:
def do_task():
pass
data = [Test(), Test(), Test()]
my_array = np.array(data, dtype=Test)
def test_hint(array:np.ndarray): # <-- I can give typehint as array but I also want to give hint about what kind of array it is.
for a in array:
a.... # Here I want Pycharm to give me list of methods from 'Test' class
I can always explicitly mention what kind of object it is like following,
for a in array:
temp = a # type: Test
temp... # Here Pycharm will give me all the suggestions.
Is there any way I can provide type-hint in the constructor so that I do not need to write additional line of code to declare type of data? I tried looking at python's type module for some clue, but couldn't find any.
I can't seem to get the shape of the tensor when I do
get_shape().as_list()
Here is the code I have written:
matrix1 = tf.placeholder(tf.int32)
matrix2 = tf.placeholder(tf.int32)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
a = sess.run(matrix1, {matrix1: [[1,2,3],[4,5,6],[7,8,9]]})
b = sess.run(matrix2, {matrix2: [[10,11,12],[13,14,15], [16,17,18]]})
print(a.get_shape().as_list()) #ERROR
I get the following error:
AttributeError: 'numpy.ndarray' object has no attribute 'get_shape'
I want to know the shape of the matrix so that I can take in an arbitrary matrix and loop through its rows and columns.
Just summarizing the discussion in the comments with few notes
Both matrix1 and a are multidimensional arrays, but there is a difference:
matrix1 is an instance of tf.Tensor, which supports two ways to access the shape: matrix1.shape attribute and matrix1.get_shape() method.
The result of tf.Tensor evaluation, a, is a numpy ndarray, which has just a.shape attribute.
Historically, tf.Tensor had only get_shape() method, shape was added later to make it similar to numpy. And one more note: in tensorflow, tensor shape can be dynamic (like in your example), in which case neither get_shape() nor shape will return a number. In this case, one can use tf.shape function to access it in runtime (here's an example when it might be useful).
I need to save and load an array, but I get this error:
cv.Save('i.xml',i)
TypeError: Cannot identify type of 'structPtr'
This is the code:
import cv
i = [[1,2],[3,4],[5,6],[7,8]]
cv.Save('i.xml',i)
That's because cv.Save needs to receive the object to be stored in the file as an OpenCV object. For example, the following is a minimal workable example that saves a numpy array in a file using cv.Save:
import cv2
import numpy as np
i = np.eye(3)
cv2.cv.Save('i.xml', cv2.cv.fromarray(i))
As you can see here, arrays should be converted back to numpy from OpenCV as well after reading.
Regards.