No Binary Operators For Structured Arrays In Numpy?

Okay, so after going through the tutorials on numpy's structured arrays I am able to create some simple examples: from numpy import array, ones names=['scalar', '1d-array', '2d-arr

Solution 1:

On the numpy structured array doc pages, most of the examples involve mixed data types - floats, ints, and strings. On SO most of the structured array questions have to do with loading mixed data from CSV files. On the other hand, in your example it appears that the main purpose of the structure is to give names to the 'columns'.

You can do math on the named columns, e.g.

struct_array1['scalar']+struct_array2['scalar']
struct_array1['2d-array']+struct_array2['2d-array']

You can also 'iterate' over the fields:

forninmy_dtype['names']:
    printa1[n]+a2[n]

And yes, for that purpose, making those arrays values in a dictionary, or attributes of an object, works just as well.

However, thinking about the CSV case, sometimes we want to talk about specific 'rows' of a CSV or structured array, e.g. struct_array[0]. Such a 'row' is a tuple of values.

In any case, the primary data structures in numpy are multiple dimensional arrays of numeric values, and most of the code revolves around number data types - float, int, etc. Structured arrays are a generalization of this, using elements that are, fundamentally, just fixed sets of bytes. How those bytes are interpreted is determined by the dtype.

Think about how MATLAB evolved - Matrices came first, then cells (like Python lists), then structures, and finally classes and objects. Python already had the lists, dictionaries and objects. numpy adds the arrays. It doesn't need to reinvent the general Python structures.

I'd lean toward defining a class like this:

class Foo(object):
    def __init__(self):
        self.scalar = 1self._1d_array = np.arange(10)
        self._2d_array = np.array([[1,2],[3,4]])

and implementing only the binary operations that really needed for the application.

Solution 2:

Another way to operate on the whole array is to use the 'union' dtype described in the documentation. In your example, you could expand your dtype by adding a 'union' field, and specifying overlapping 'offsets':

from numpy import array, ones, zeros

names=['scalar', '1d-array', '2d-array', 'union']
formats=['float64', '(3,)float64', '(2,2)float64', '(8,)float64']
offsets=[0, 8, 32, 0]
my_dtype = dict(names=names, formats=formats, offsets=offsets)
struct_array3=zeros((4,), dtype=my_dtype)

['union'] now gives access to all the data as a (n,8) array

struct_array3['union'] # == struct_array3.view('(8,)f8')
struct_array3['union'].shape  # (4,8)

You can operate on 'union' or any other fields:

struct_array3['union'] += 2
struct_array3['scalar']= 1

The 'union' field could another compatible shape, such as '(2,4)float64'. A 'row' of such an array might look like:

array([ (3.0, [0.0, 0.0, 0.0], [[2.0, 2.0], [0.0, 0.0]], 
      [[3.0, 0.0, 0.0, 0.0], [2.0, 2.0, 0.0, 0.0]])], 
      dtype={'names':['scalar','1d-array','2d-array','union'], 
             'formats':['<f8',('<f8', (3,)),('<f8', (2, 2)),('<f8', (2, 4))], 
             'offsets':[0,8,32,0], 
             'itemsize':64})

Solution 3:

Okay, after more research I stumbled upon an answer. (No fault to hpaulj - the question was not posed all that well.) But I wanted to post in case someone else has a similar frustration.

The answer comes from the numpy documentation on ndarray.view. They specifically provide an example in which they "[create] a view on a structured array so it can be used in calculations".

So, I was frustrated that I couldn't operate on my example structured arrays. After all, I "see" my structured array as simply a collection of floating point numbers! Well, in the end all I needed was to inform numpy of this abstraction using "view". The errors in the question can be avoided using:

( struct_array1.view(dtype='float64') + struct_array2.view(dtype='float64') ).view(dtype=my_dtype)
( struct_array1.view(dtype='float64') + struct_array2.view(dtype='float64') ).view(dtype=my_dtype)
( 1.0 + struct_array2.view(dtype='float64') ).view(dtype=my_dtype)
( 2.0 * struct_array2.view(dtype='float64') ).view(dtype=my_dtype)

This is not as elegant as one might want, but at least numpy has the capability.