Orange Forum • View topic - Create table from a large Scipy Sparse matrix

Create table from a large Scipy Sparse matrix

A place to ask questions about methods in Orange and how they are used and other general support.

Create table from a large Scipy Sparse matrix

Postby nawaf » Fri Jan 17, 2014 14:24

I've got a scipy sparse matrix (csr:Compressed Sparse Row matrix). I'd like to use Orange's feature selection methods (Orange.feature.scoring.score_all (InfoGain/MDL)). However, from my understanding I'll have to create a Table which only accepts a numpy array as an arguments. Therefore, whenever I tried to convert the csr matrix to an array, using (.toarray()), I get the following error (because the size of the matrix):

Code: Select all
Traceback (most recent call last):
  File "C:\Users\NMS\Desktop\PyExp\experiments_acl2013.py", line 249, in <module>
    print(X_train.toarray())
  File "C:\Python27\lib\site-packages\scipy\sparse\compressed.py", line 561, in toarray
    return self.tocoo(copy=False).toarray(order=order, out=out)
  File "C:\Python27\lib\site-packages\scipy\sparse\coo.py", line 238, in toarray
    B = self._process_toarray_args(order, out)
  File "C:\Python27\lib\site-packages\scipy\sparse\base.py", line 635, in _process_toarray_args
    return np.zeros(self.shape, dtype=self.dtype, order=order)
ValueError: array is too big.


Is there another approach that can allow me to pass a sparse matrix to create a table?
OR
Is there a way to apply InfoGain or MDL, in Orange, without creating a table using my sparse matrix directly?

Re: Create table from a large Scipy Sparse matrix

Postby Ales » Tue Jan 21, 2014 14:38

nawaf wrote:Is there another approach that can allow me to pass a sparse matrix to create a table?
Unfortunately no.

nawaf wrote:Is there a way to apply InfoGain or MDL, in Orange, without creating a table using my sparse matrix directly?
No.


Return to Questions & Support