|
Difference between
version 7
and
version 6:
| Line 6 was replaced by line 6 |
| - # The Ptolemy array of tokens is inefficient while the Ptolemy Matrix is designed for 2-D info (images). I suggest that we add a new multidimensional array type to Kepler. Data would be stored linearly in memory (1-D Java array) with a second array indicating dimensionality (approach used in R). Thus a 1-D array with 12 elements could be dimensioned as a 1x12, a 2x6, 3x4, 2x3x2 etc arrays and the dimensionality can changed without moving any data. \\ \\(This is in effect the way that OPeNDAP stores it's array data in memory. If care was taken designing this Token type we could allow the internal storage to be passed in - thus allowing us to read OPeNDAP data and seamlessly wrap it in a Token without being forced to do an element by element copy. ndp 11/14/07)\\ \\ |
| + # The Ptolemy array of tokens is inefficient while the Ptolemy Matrix is designed for 2-D info (images). I suggest that we add a new multidimensional array type to Kepler. Data would be stored linearly in memory (1-D Java array) with a second array indicating dimensionality (approach used in R). Thus a 1-D array with 12 elements could be dimensioned as a 1x12, a 2x6, 3x4, 2x3x2 etc arrays and the dimensionality can changed without moving any data. \\ \\{{(This is in effect the way that OPeNDAP stores it's array data in memory. If care was taken designing this Token type we could allow the internal storage to be passed in - thus allowing us to read OPeNDAP data and seamlessly wrap it in a Token without being forced to do an element by element copy. ndp 11/14/07)}}\\ \\ |
| At line 10 added 2 lines. |
| + ;__Comments from Nathan Potter - 11/15/2007__:'' I have added an optimization step to the OPeNDAP actor in which it "squeezes" incoming arrays to remove dimensions whose size is equal to one. The result is that if the user subsets an N-Dimensional array in such a way that the result is effectively a 1 or 2 dimensional array, then the Actor will map it to a matrix (1xN and MxN respectively). While this doesn't address all of the memory usage concerns it will enable us to move forward with workflow development for the SST use case. That work should bring more focus on the other memory limitations that we may encounter. |
| + |
Back to O Pe NDAP Kepler Data Model Resolution,
or to the Page History.
|
