Date: Thu, 10 Jan 2008 15:28:55 -0500
Reply-To: Randall Powers <powers_r@BLS.GOV>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: Randall Powers <powers_r@BLS.GOV>
Subject: Re: Can you do array processing and a proc nlp from within proc
iml?
Content-Type: text/plain; charset=ISO-8859-1
First, thanks to data_null, whose questions helped me figure out what my
problem was. And to Mary, whose suggestion helped as well.
I have one more question that needs to be answered before this phase of the
program is complete.
My PROC IML uses
prob = {0.232,0.16,0.16,0.16,0.16,0.032,0.032,0.032,0.032}; as input for my
RANDMULTINOMAL function. (see program below). What I'd like it to use
instead of these manually input values which are subject to change
frequently is prob = {PIE0, PIE1, PIE2, PIE3, PIE4, PIE5, PIE6, PIE7,
PIE8}; using the variables PIE0-PIE8 generated by the data step out.PieCalc
(see below).
I have fiddled around a bit with the USE, LIST, and READ statements for
PROC IML (below) but I'm still getting this error. Can anybody tell me how
I would correct this problem? Note: when I use the line with the actual
numeric values (and comment out the like with prob={PIE0...}), the program
runs fine.
Thanks!
The error message:
7473 proc iml;
NOTE: IML Ready
7474 use out.PIECALC var{PIE0 PIE1 PIE2 PIE3 PIE4 PIE5 PIE6 PIE7 PIE8};
7475 load module=RANDMULTINOMIAL;
NOTE: Opening storage library WORK.IMLSTOR
7476 list all ;
7477 read all;
7478 call randseed(1);
7479 prob = {PIE0, PIE1, PIE2, PIE3, PIE4, PIE5, PIE6, PIE7, PIE8};
7480 /*prob = {0.232,0.16,0.16,0.16,0.16,0.032,0.032,0.032,0.032};*/
7481 NumTrials = 50;
7482 N = 10;
7483 x = RANDMULTINOMIAL(N,NumTrials,prob);
NOTE: Module ROWVEC loaded from the storage SASHELP.IMLMLIB.
ERROR: (execution) Character argument should be numeric.
operation : SUM at offset 26 column 19
operands : PROB
PROB 9 rows 1 col (character, size 4)
PIE0
PIE1
PIE2
PIE3
PIE4
PIE5
PIE6
PIE7
PIE8
statement : IF at offset 26 column 5
traceback : module RANDMULTINOMIAL at offset 26 column 5
NOTE: Paused in module RANDMULTINOMIAL.
7484
7485 create out.testset from x ;
ERROR: Matrix X has not been set to a value.
statement : CREATE at line 7485 column 1
7486 append from x;
ERROR: No data set is currently open for output.
statement : APPEND at line 7486 column 1
7487
7488 print x;
ERROR: Matrix X has not been set to a value.
statement : PRINT at line 7488 column 1
7489
7490
7491 quit;
NOTE: Exiting IML.
NOTE: Storage library WORK.IMLSTOR closed.
NOTE: The SAS System stopped processing this step because of errors.
NOTE: PROCEDURE IML used (Total process time):
THE PROGRAM:
LibName out 'C:\test';
data out.PieCalc;
array PIE{2,4} PIE1-PIE8;
array q{4} q1-q4;
array f{4} f1-f4;
array piezerosum{4} piezerosum1-piezerosum4;
array n{9} n1-n9;
q1=.8;
q2=.8;
q3=.8;
q4=.8;
gamma=.8;
f1=.25;
f2=.25;
f3=.25;
f4=.25;
RR=2;
M=2;
do r=1 to 2;
do k=1 to 4;
PIE(r,k)=gamma*((1-q(k))**(r-1))*q(k)*f(k);
piezerosum(k)=(1-q(k))**RR*f(k);
end;
end;
piezerosumtot=sum(of piezerosum1-piezerosum4);
PIE0= (1-gamma)+gamma*piezerosumtot;
run;
proc print data=out.PieCalc;
run;
proc iml;
use out.PIECALC var{PIE0 PIE1 PIE2 PIE3 PIE4 PIE5 PIE6 PIE7 PIE8};
load module=RANDMULTINOMIAL;
list all ;
read all;
call randseed(1);
prob = {PIE0, PIE1, PIE2, PIE3, PIE4, PIE5, PIE6, PIE7, PIE8};
/*prob = {0.232,0.16,0.16,0.16,0.16,0.032,0.032,0.032,0.032};*/
NumTrials = 50;
N = 10;
x = RANDMULTINOMIAL(N,NumTrials,prob);
create out.testset from x ;
append from x;
print x;
quit;
On Wed, 9 Jan 2008 10:38:48 -0600, data _null_, <datanull@GMAIL.COM> wrote:
>1) How many rows are in the maxtix that is output from RANDMULTINOMIAL?
>2) What are the columns names in the data set created, thay need to be
>made to match the names you are using in PROC NLP.
>3) How many rows from RANDMULTINOMIAL are needed for each "fit" using PROC
NLP.
>4) What is _OVERALL for?
>
>On Jan 9, 2008 9:42 AM, Randall Powers <powers_r@bls.gov> wrote:
>> Thanks again, that sounds promising. The log is giving me the follow
error
>> (below, each iteration has the same error messages). My program is also
>> below. What do I need to alter to correct the errors? Previously, I had
>> manual inputs of n0, n11....n24, and the program worked. So it's
obviously
>> just a question of somehow letting SAS know that the values in the
dataset
>> are the values of n0, n11....n24.
>>
>>
>> NOTE: Your code contains 5 program statements.
>> NOTE: Gradient is computed using analytic formulas.
>> NOTE: Hessian is computed using analytic formulas.
>> ERROR: The variable n11 was referenced but not given a value.
>> ERROR: The variable n12 was referenced but not given a value.
>> ERROR: The variable n13 was referenced but not given a value.
>> ERROR: The variable n14 was referenced but not given a value.
>> ERROR: The variable n21 was referenced but not given a value.
>> ERROR: The variable n22 was referenced but not given a value.
>> ERROR: The variable n23 was referenced but not given a value.
>> ERROR: The variable n24 was referenced but not given a value.
>> ERROR: The variable n0 was referenced but not given a value.
>> WARNING: Your program statements contain 9 symbols used but not given a
>> value.
>> WARNING: None of the data set variables are used in the program
statements.
>> NOTE: Initial value of parameter gamma is set randomly to 0.7504856474.
>> NOTE: Initial value of parameter q1 is set randomly to 0.6170781951.
>> NOTE: Initial value of parameter q2 is set randomly to 0.4308046808.
>> NOTE: Initial value of parameter q3 is set randomly to 0.9153506276.
>> NOTE: Initial value of parameter q4 is set randomly to 0.7203864049.
>> WARNING: Your program statements cannot be executed completely.
>> WARNING: In a total of 1 calls an error occurred during execution of the
>> program statements. NLP attempted to recover by using a
>> shorter step size.
>> NOTE: View OUT.FORNLP.VIEW used (Total process time):
>> real time 0.18 seconds
>> cpu time 0.15 seconds
>>
>> NOTE: PROCEDURE NLP used (Total process time):
>> real time 0.20 seconds
>> cpu time 0.17 seconds
>>
>>
>>
>>
>>
>>
>> LibName out 'C:\test';
>> data out.forNLP / view=out.forNLP;
>> set out.testset3; *the output data set from RANDMULTINOMIAL;
>> replicate = _n_;
>> run;
>>
>> %macro _overall(_itcnt=500);
>> %do _i=1 %to &_itcnt;
>>
>> proc NLP data=out.forNLP;
>> by replicate;
>> *the rest;
>>
>>
>> f1=.25;f2=.25;f3=.25;f4=.25;
>> max h;
>> decvar gamma q1 q2 q3 q4;
>> bounds 0<=gamma<=1;
>> bounds 0<=q1<=1;
>> bounds 0<=q2<=1;
>> bounds 0<=q3<=1;
>> bounds 0<=q4<=1;
>>
>>
>>
>> h=n11*log(f1*gamma*q1)+n12*log(f2*gamma*q2)+n13*log(f3*gamma*q3)+n14*log
>> (f4*gamma*q4)+n21*log(f1*q1*gamma-f1*q1*q1*gamma)+n22*log(f2*q2*gamma-
>> f2*q2*q2*gamma)+n23*log(f3*q3*gamma-f3*q3*q3*gamma)+n24*log(f4*q4*gamma-
>> f3*q4*q4*gamma)+n0*log((1-gamma)+gamma*((1-q1)*(1-q1)*f1+(1-q2)*(1-q2)
*f2+
>> (1-q3)*(1-q3)*f3+(1-q4)*(1-q4)*f4));
>> run;
>> %end;
>>
>> %mend _overall;
>>
>> %_overall;
>>
>>
>>
>>
>>
>>
>>
>> On Wed, 9 Jan 2008 08:41:13 -0600, data _null_, <datanull@GMAIL.COM>
wrote:
>>
>> >I think you need to use the BY statement in PROC NLP.
>> >
>> >after you create the output data set from RANDMUNTINOMIAL use a data
>> >step view to create a replicate variable then run proc nlp by
>> >replicate;
>> >
>> >data work.forNLP / view=work.forNLP;
>> > set work.x; *the output data set from RANDMULTINOMIAL;
>> > replicate = _n_;
>> > run;
>> >
>> >proc NLP data=work.forNLP;
>> > by replicate;
>> > *the rest;
>> > run;
>> >
>> >
>> >On Jan 9, 2008 8:20 AM, Randall Powers <powers_r@bls.gov> wrote:
>> >> Thanks to all who have been replying. Maybe I haven't been clear
enough
>> as
>> >> to what I'm trying to do. So I'll try to explain again.
>> >>
>> >> I have a function h that I'm using PROC NLP to maximize:
>> >>
>> >> h=n11*log(f1*gamma*q1)+n12*log(f2*gamma*q2)+n13*log(f3*gamma*q3)
+n14*log
>> >> (f4*gamma*q4)+n21*log(f1*q1*gamma-f1*q1*q1*gamma)+n22*log(f2*q2*gamma-
>> >> f2*q2*q2*gamma)+n23*log(f3*q3*gamma-f3*q3*q3*gamma)+n24*log
(f4*q4*gamma-
>> >> f3*q4*q4*gamma)+n0*log((1-gamma)+gamma*((1-q1)*(1-q1)*f1+(1-q2)*(1-q2)
>> *f2+
>> >> (1-q3)*(1-q3)*f3+(1-q4)*(1-q4)*f4));
>> >>
>> >> The decision variables are:
>> >> gamma q1 q2 q3 q4
>> >>
>> >> I assign PROC NLP the values of f1,f2,f3,f4.
>> >>
>> >> I can also assign PROC NLP the values of n0, n11, n12, n13, n14, n21,
>> n22,
>> >> n23, n24. However, I want to run this nlp many (let's say P) times,
so I
>> am
>> >> using PROC IML to generate a matrix of values:
>> >>
>> >> n0 n11 n12 n13 n14 n21 n22 n23 n24
>> >> n0 n11 n12 n13 n14 n21 n22 n23 n24
>> >> .
>> >> .
>> >> .
>> >> n0 n11 n12 n13 n14 n21 n22 n23 n24 (Pth row)
>> >>
>> >> to use as input.
>> >>
>> >> Thus, each row of this matrix are the variables to a maximization
>> problem
>> >> that I wish to run P times. So I need a matrix of P rows and (in this
>> case,
>> >> though it may change depending on what I'm maximizing) 9 columns (I
made
>> my
>> >> previous example the the list a 10x3 matrix, for sake of keeping it
>> simple.)
>> >>
>> >> The values of n0, n11, n12, n13, n14, n21, n22, n23, n24 are
determined
>> >> using the proc iml statement below. For this example, we would get a
10x9
>> >> matrix, where the values in each row would add up to 50, and the
>> >> probability of randomly getting each of the values n0, n11, n12, n13,
>> n14,
>> >> n21, n22, n23, n24 would be 0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.2,
>> >> respectively.
>> >>
>> >>
>> >> proc iml;
>> >> load module=RANDMULTINOMIAL;
>> >> /* doc example */
>> >> call randseed(1);
>> >> prob = {0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.2};
>> >> NumTrials = 50;
>> >> N = 10;
>> >> x = RANDMULTINOMIAL(N,NumTrials,prob);
>> >> create out.testset2 from x ;
>> >> append from x;
>> >> print x;
>> >> quit;
>> >>
>> >> Thus, my hope is to take the 10x9 (or 10x3 in my previous example, or
>> >> Pxwhatever) and use it as an array of inputs, with each row being a
>> >> separate collection of variables for a single PROC NLP run.
>> >>
>> >> Thanks again to whoever can make heads or tails of this and offer
>> sensible
>> >> ideas!
>> >>
>> >> On Tue, 8 Jan 2008 15:39:04 -0600, data _null_, <datanull@GMAIL.COM>
>> wrote:
>> >>
>> >> >I thought you wanted to run PROC NLP. What is the array for?
>> >> >
>> >> >On Jan 8, 2008 3:31 PM, Randall Powers <powers_r@bls.gov> wrote:
>> >> >> Ok, that gives me a data set with 10 rows and 3 columns.
>> >> >>
>> >> >> Perhaps a dumb question, but how do I now get this dataset into a
two
>> >> >> dimemsional 10x3 array?
>> >> >>
>> >> >> Thanks again!
>> >> >>
>> >> >>
>> >> >> On Tue, 8 Jan 2008 14:10:17 -0600, data _null_,
<datanull@GMAIL.COM>
>> >> wrote:
>> >> >>
>> >> >> >Try the FROM clause
>> >> >> >
>> >> >> >create out.testset2 from x;
>> >> >> >append from x;
>> >> >> >
>> >> >> >
>> >> >> >
>> >>
>> >> >> >On Jan 8, 2008 1:26 PM, Randall Powers <powers_r@bls.gov> wrote:
>> >> >> >> Thanks, Datanull.
>> >> >> >>
>> >> >> >> I was asking if there was a way to convert the matrix into an
array
>> >> >> >> directly within proc iml. If this is not possible, then I would
go
>> the
>> >> >> >> route you suggest...
>> >> >> >>
>> >> >> >> Indeed, I had thought to do that and had gotten that far. Take
the
>> >> >> matrix,
>> >> >> >> create a dataset (that's as far as I'd gotten), then assign it
>> into an
>> >> >> >> array.
>> >> >> >>
>> >> >> >> One kink I'm noticing (see example below)... This code produces
a
>> 10x3
>> >> >> >> matrix where each row adds up to 50. However, when I use the
create
>> >> and
>> >> >> >> append statements (and a subsequent data step to rid myself of
all
>> >> other
>> >> >> >> variables), it creates a dataset that is 30x1, rather than the
>> >> promised
>> >> >> >> 10x3 dataset.
>> >> >> >>
>> >> >> >> So two questions:
>> >> >> >>
>> >> >> >> 1. How do I get it to create a 10x3 dataset which mimics my
>> matrix, as
>> >> >> >> promised?
>> >> >> >>
>> >> >> >> OR
>> >> >> >>
>> >> >> >> 2. How do I read the 30x1 dataset into my desired 10x3 array?
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> proc iml;
>> >> >> >> load module=RANDMULTINOMIAL;
>> >> >> >> call randseed(1);
>> >> >> >> prob = {0.3,0.6,0.1};
>> >> >> >> NumTrials = 50;
>> >> >> >> N = 10;
>> >> >> >> x = RANDMULTINOMIAL(N,NumTrials,prob);
>> >> >> >> create out.testset2 ;
>> >> >> >> append;
>> >> >> >> print x;
>> >> >> >> quit;
>> >> >> >>
>> >> >> >> data out.testset3;
>> >> >> >> set out.testset2 (keep=x);
>> >> >> >> run;
>> >> >> >>
>> >> >> >>
>> >> >> >> SAS proc iml output:
>> >> >> >>
>> >> >> >>
>> >> >> >> 14 26 10
>> >> >> >> 13 30 7
>> >> >> >> 13 28 9
>> >> >> >> 19 25 6
>> >> >> >> 15 27 8
>> >> >> >> 15 29 6
>> >> >> >> 14 27 9
>> >> >> >> 20 27 3
>> >> >> >> 18 26 6
>> >> >> >> 11 32 7
>> >> >> >>
>> >> >> >> SAS dataset out.testset3:
>> >> >> >>
>> >> >> >> 14
>> >> >> >> 26
>> >> >> >> 10
>> >> >> >> 13
>> >> >> >> 30
>> >> >> >> 7
>> >> >> >> 13
>> >> >> >> 29
>> >> >> >> 9
>> >> >> >> .
>> >> >> >> .
>> >> >> >> .
>> >> >> >> 11
>> >> >> >> 32
>> >> >> >> 7
>> >> >> >>
>> >> >> >> On Tue, 8 Jan 2008 10:53:23 -0600, data _null_,
>> <datanull@GMAIL.COM>
>> >> >> wrote:
>> >> >> >>
>> >> >> >> >I think, you want to create a SAS data set from a matrix...
>> >> >> >> >
>> >> >> >> >Creating a SAS Data Set from a Matrix
>> >> >> >> >SAS/IML software provides the ability to create a new SAS data
set
>> >> >> >> >from a matrix. Use the CREATE and APPEND statements to create a
>> SAS
>> >> >> >> >data set from a matrix, where the columns of the matrix become
the
>> >> >> >> >data set variables and the rows of the matrix become the
>> >> observations.
>> >> >> >> >Thus, an n matrix creates a SAS data set with m variables
and n
>> >> >> >>
>> >> >> >> >observations. The CREATE statement opens the new SAS data set
for
>> >> both
>> >> >> >> >input and output, and the APPEND statement writes to (outputs
to)
>> the
>> >> >> >> >data set.
>> >> >> >> >
>> >> >> >> >Then use that data set as input to PROC NLP.
>> >> >> >> >
>> >> >> >> >On Jan 8, 2008 10:30 AM, Randall Powers <powers_r@bls.gov>
wrote:
>> >> >> >> >> Sorry this is rambling and longwinded. Any input anybody can
>> give
>> >> >> would
>> >> >> >> be
>> >> >> >> >> helpful! If what I'm saying/asking doesn't make sense, it's
>> >> probably
>> >> >> >> >> because of my newbieness...I'd be happy to clarify.
>> >> >> >> >>
>> >> >> >> >> So here is what I'm doing.
>> >> >> >> >>
>> >> >> >> >> I'm using proc iml to create a matrix of values. The number
of
>> >> columns
>> >> >> >> and
>> >> >> >> >> rows may vary, but for arguments sake, let's say 50 rows and
5
>> >> >> columns.
>> >> >> >> I'm
>> >> >> >> >> using RANDMULTIMOMIAL to create matrix X, which is 50x5. What
>> I'd
>> >> >> like to
>> >> >> >> >> do is somehow convert that matrix into a 50x5 array (50
rows, 5
>> >> >> columns),
>> >> >> >> >> which I would then use as input in a proc nlp.
>> >> >> >> >>
>> >> >> >> >> For example:
>> >> >> >> >>
>> >> >> >> >> 5 4 6 2 5
>> >> >> >> >> 4 5 4 4 5
>> >> >> >> >> 3 7 3 6 3
>> >> >> >> >> .........
>> >> >> >> >> .........
>> >> >> >> >> etc
>> >> >> >> >>
>> >> >> >> >> Does this seem possible? Can it all be done in the proc iml,
or
>> do
>> >> I
>> >> >> have
>> >> >> >> >> to break it up and do it outside of proc iml?
>> >> >> >> >>
>> >> >> >> >> One thing I did try was to create an output file which I'm
>> thinking
>> >> >> could
>> >> >> >> >> then be used to read the values of X into an array in a data
>> step.
>> >> The
>> >> >> >> >> output file created from proc iml assigns each value of the
>> matrix
>> >> a
>> >> >> >> >> separate observation. Thus, the example below gives a SAS
data
>> set
>> >> >> with:
>> >> >> >> >>
>> >> >> >> >> OBS X
>> >> >> >> >> 1 5
>> >> >> >> >> 2 4
>> >> >> >> >> 3 6
>> >> >> >> >> 4 2
>> >> >> >> >> 5 5
>> >> >> >> >> 6 4
>> >> >> >> >> 7 5
>> >> >> >> >> 8 4
>> >> >> >> >> 9 4
>> >> >> >> >> 10 5
>> >> >> >> >> etc. (in this example, I would have 50x5=250 obs)
>> >> >> >> >>
>> >> >> >> >> How would I then capture the value of X for the first five
>> >> >> observations
>> >> >> >> >> into an array, the second five values into another array, the
>> third
>> >> >> five
>> >> >> >> >> values into a third array, etc. so that I would have 50
arrays
>> of
>> >> five
>> >> >> >> >> values (or a multidimensional array of size {50,5} containing
>> all
>> >> of
>> >> >> the
>> >> >> >> >> values? I would then send this array to my proc nlp, where it
>> >> would be
>> >> >> >> used.
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>>
| 
