Variable Scope

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Variable Scope

Simon Forster
Folks

Can someone explain Lasso's variable scope to me as I've just been  
bitten by this. I'd like a general overview (which I thought I had  
but apparently not). My specific circumstance is:

- Process. Creates a map of data.

- Map of data is passed into an async custom tag.

- Async tag communicates with remote servers and eventually drops  
some response data into a database, based on the keyfield contained  
within the map of data initially passed in.

- Meanwhile, parent process is updating the map of data to be passed  
into the next (async) thread.

Issue: Basically, the data passed into the async tag seems to be  
updated by the parent process. My reading of the docs suggested that  
an async thread is its own, self contained little world - which is  
why you have to explicitly pass in the variable data. However, this  
understanding seems to be wrong. i.e. The variables seem to be passed  
in by reference rather than copied.

The work around is quite simple, make copies of the map of data and  
pass the copy into the async tag:

var: ('data' loop_count) = $data;

async_tag: -data = var: ('data' loop_count), ...

However, this would appear to be quite clumsy and potentially is a  
memory leak waiting to happen (we're killing off the parent process  
after 10 iterations so, in this instance, I'm not too concerned).  
What I am worried about is my misunderstanding of how Lasso manages  
variables, especially with reference to asynchronous processes.

Mac OS Server 10.4.7, Lasso 8.1.

TIA for the enlightenment.


------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Variable Scope

Bil Corry-3
Simon Forster wrote:
> However, this would appear to be quite clumsy and potentially is a
> memory leak waiting to happen (we're killing off the parent process
> after 10 iterations so, in this instance, I'm not too concerned). What I
> am worried about is my misunderstanding of how Lasso manages variables,

Lasso passes parameters to tags by reference, unless you tell it otherwise:

  define_tag:'test',
    -required='hello',
    -required='world';
    // #hello and #world are references to original values
  define_tag;

vs.

  define_tag:'test',
    -required='hello', -copy
    -required='world', -copy;
    // #hello and #world are copies of the original values
  define_tag;


I've never tested an async tag to see if it holds true, but that's how regular ctags work and I'd imagine -async tags as well.


- Bil


------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Variable Scope

Simon Forster

On 1 Sep 2006, at 14:23, Bil Corry wrote:

> Simon Forster wrote:
>> However, this would appear to be quite clumsy and potentially is a  
>> memory leak waiting to happen (we're killing off the parent  
>> process after 10 iterations so, in this instance, I'm not too  
>> concerned). What I am worried about is my misunderstanding of how  
>> Lasso manages variables,
>
> Lasso passes parameters to tags by reference, unless you tell it  
> otherwise:
>
>  define_tag:'test',
>    -required='hello',
>    -required='world';
>    // #hello and #world are references to original values
>  define_tag;
>
> vs.
>
>  define_tag:'test',
>    -required='hello', -copy
>    -required='world', -copy;
>    // #hello and #world are copies of the original values
>  define_tag;
>
>
> I've never tested an async tag to see if it holds true, but that's  
> how regular ctags work and I'd imagine -async tags as well.

I'll have to test - when I'm not up against a deadline.

Thanks Bil.

Simon Forster
_______________________________________________________
  LDML Ltd, 62 Pall Mall, London, SW1Y 5HZ, UK
  Tel: +44 (0)20 8133 0528   Fax: +44 (0)70 9230 5247
_______________________________________________________



------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Variable Scope

Simon Forster
In reply to this post by Bil Corry-3

On 1 Sep 2006, at 14:23, Bil Corry wrote:

> Lasso passes parameters to tags by reference, unless you tell it  
> otherwise:
>
>  define_tag:'test',
>    -required='hello',
>    -required='world';
>    // #hello and #world are references to original values
>  define_tag;
>
> vs.
>
>  define_tag:'test',
>    -required='hello', -copy
>    -required='world', -copy;
>    // #hello and #world are copies of the original values
>  define_tag;
>
>
> I've never tested an async tag to see if it holds true, but that's  
> how regular ctags work and I'd imagine -async tags as well.

OK. Added in the -copy to the define tag:

define_tag:
        'async',
        -namespace = 'ldml_',
        -required = 'inc',
        -copy,
        -type = 'string',
        -optional = 'data',
        -copy,
        -type = 'map',
        -optional = 'debug',
        -copy,
        -type = 'boolean',
        -async,
        -description= 'A simple wrapper to force a task to go asynchronous';
       
        include: #inc;
       
/define_tag;

Originally I had the -copy after the type declaration but that didn't  
work either so I chucked it straight after the -required and -
optional declarations. No difference.

I'll revert to my previous set-up but I'd like to get to the bottom  
of this 'cause I've set up the potential for a memory leak.

TIA for any further suggestions (other than use the {...}->async;  
construct!).

Simon Forster
_______________________________________________________
  LDML Ltd, 62 Pall Mall, London, SW1Y 5HZ, UK
  Tel: +44 (0)20 8133 0528   Fax: +44 (0)70 9230 5247
_______________________________________________________



------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Variable Scope

decorior
Isn't a comma needed after the first copy?

Deco
On Sep 6, 2006, at 5:40 AM, Simon Forster wrote:

>
> On 1 Sep 2006, at 14:23, Bil Corry wrote:
>
>> Lasso passes parameters to tags by reference, unless you tell it  
>> otherwise:
>>
>>  define_tag:'test',
>>    -required='hello',
>>    -required='world';
>>    // #hello and #world are references to original values
>>  define_tag;
>>
>> vs.
>>
>>  define_tag:'test',
>>    -required='hello', -copy
>>    -required='world', -copy;
>>    // #hello and #world are copies of the original values
>>  define_tag;
>>
>>
>> I've never tested an async tag to see if it holds true, but that's  
>> how regular ctags work and I'd imagine -async tags as well.
>
> OK. Added in the -copy to the define tag:
>
> define_tag:
> 'async',
> -namespace = 'ldml_',
> -required = 'inc',
> -copy,
> -type = 'string',
> -optional = 'data',
> -copy,
> -type = 'map',
> -optional = 'debug',
> -copy,
> -type = 'boolean',
> -async,
> -description= 'A simple wrapper to force a task to go asynchronous';
>
> include: #inc;
>
> /define_tag;
>
> Originally I had the -copy after the type declaration but that  
> didn't work either so I chucked it straight after the -required and  
> -optional declarations. No difference.
>
> I'll revert to my previous set-up but I'd like to get to the bottom  
> of this 'cause I've set up the potential for a memory leak.
>
> TIA for any further suggestions (other than use the {...}->async;  
> construct!).
>
> Simon Forster
> _______________________________________________________
>  LDML Ltd, 62 Pall Mall, London, SW1Y 5HZ, UK
>  Tel: +44 (0)20 8133 0528   Fax: +44 (0)70 9230 5247
> _______________________________________________________
>
>
>
> ------------------------------
> Lasso Support: http://support.omnipilot.com/
> Search the list archives: http://www.listsearch.com/lassotalk.lasso
> Manage your list subscription: http://www.listsearch.com/ 
> lassotalk.lasso?manage


------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Variable Scope

Simon Forster

On 6 Sep 2006, at 13:00, Deco Rior wrote:

> Isn't a comma needed after the first copy?

I assumed a small typo from Bil. You'll see commas in place for my code.

I hope the -copy flag works in normal custom tags 'cause otherwise  
people could have quite an insidious bug. Actually, I don't  
understand why values aren't copied in by default for async tags -  
all seems counter-intuitive to me - but no doubt I'm missing an  
important point here.

Simon Forster
_______________________________________________________
  LDML Ltd, 62 Pall Mall, London, SW1Y 5HZ, UK
  Tel: +44 (0)20 8133 0528   Fax: +44 (0)70 9230 5247
_______________________________________________________



------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Variable Scope

decorior
I guess under certain circumstances you might want the async process  
to use the current value? In which case, following a similar  
structure could make sense. Also, I would think this makes the lasso  
development code easier, since it is not dealing with different  
behaviors based on the existence of another tag.


On Sep 6, 2006, at 6:20 AM, Simon Forster wrote:

>
> On 6 Sep 2006, at 13:00, Deco Rior wrote:
>
>> Isn't a comma needed after the first copy?
>
> I assumed a small typo from Bil. You'll see commas in place for my  
> code.
>
> I hope the -copy flag works in normal custom tags 'cause otherwise  
> people could have quite an insidious bug. Actually, I don't  
> understand why values aren't copied in by default for async tags -  
> all seems counter-intuitive to me - but no doubt I'm missing an  
> important point here.
>
> Simon Forster
> _______________________________________________________
>  LDML Ltd, 62 Pall Mall, London, SW1Y 5HZ, UK
>  Tel: +44 (0)20 8133 0528   Fax: +44 (0)70 9230 5247
> _______________________________________________________
>
>
>
> ------------------------------
> Lasso Support: http://support.omnipilot.com/
> Search the list archives: http://www.listsearch.com/lassotalk.lasso
> Manage your list subscription: http://www.listsearch.com/ 
> lassotalk.lasso?manage


------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Variable Scope

Fletcher Sandbeck
In reply to this post by Simon Forster
On 2006-09-06 at 13:20 by [hidden email] (Simon Forster):

>I hope the -copy flag works in normal custom tags 'cause otherwise  
>people could have quite an insidious bug. Actually, I don't  
>understand why values aren't copied in by default for async tags -  
>all seems counter-intuitive to me - but no doubt I'm missing an  
>important point here.

Lasso passes all values by reference by default.  This includes values passed to custom tags as well as to built-in tags.  When running a program memory use generally corresponds directly to time.  The less memory a program uses the faster it runs.  Every time a value is copied it takes twice as much memory and the copy itself takes time to complete.

Every value in Lasso has a reference count.  This is basically a count of how many variable names are pointing to the value.  The value will remain in memory until the reference count goes down to 0.  You can get the current reference count of a value using the [null->refcount] tag.  The following code returns 2.  I think one reference is maintained internally and the second is the actual variable.

[var: 'myvar' = 'myvalue']
[$myvar->refcount] -> 2

If we assign a reference to the variable to another variable then the refcount goes up by one.

[var: 'myref' = @$myvar]
[$myvar->refcount] -> 3

If we pass the variable into a custom tag then the refcount goes up by two.  I believe one reference is used by the [Params] array and the second by the local representing the parameter.

[var: 'myvar' = 'myvalue']
[define_tag: 'test1', -required='value']
  [return: #value->refcount]
[/define_tag]
[test1: $myvar] -> 4

If you -Copy the value into the custom tag then a new value is generated and will only have a refcount of 2.  You can see this by adding -Copy to the [Define_Tag] above.  

If you use -Async tags the results are similar, but if you insert a [Sleep] that is long enough you can actually see Lasso decrement the refcount as the page is destroyed.  The following code logs 3 after one second has passed.

[define_tag: 'test1', -required='value', -async]
  [sleep: 1000]
  [log_always: 'one ' + #value->refcount]
[/define_tag]
[test1: $myvar]

Anyway, long story short, your code shouldn't cause a memory leak because Lasso will clean up the variables once your page completes processing and the asynchronous process completes processing.  At that point the refcount drops to zero and the value is destroyed.

[fletcher]
--
Fletcher Sandbeck                         [hidden email]
Director of Product Development       http://www.lassostudio.com
OmniPilot Software, Inc.                http://www.omnipilot.com

------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Variable Scope

Simon Forster

On 6 Sep 2006, at 17:39, Fletcher Sandbeck wrote:

> Every value in Lasso has a reference count.

I guessed this is how the garbage collector would be working - in  
fact it seems to be the universally accepted method.

> If you -Copy the value into the custom tag then a new value is  
> generated and will only have a refcount of 2.

This doesn't seem to be working for me (Mac OS Server 10.4.7, Lasso  
8.1). I have an -async tag to which I add -copy for the -required and  
-optional parameters. Parent process spawns an -async process. Child  
(-async) process communicates with remote server and waits for reply.  
Meanwhile, parent process sets up variables for the next -async  
process. By the time the replies come in from the remote server, the  
key id passed in to the -async processes has been modified by the  
parent process and only the record associated with the last -async  
process gets updated. i.e. The -copy parameter seems to make no  
difference - the variables have still been passed in by reference.

Workaround: Assign data to a unique variable in the parent process  
and pass in the unique variable to the -async tag.

Potential issue: If the parent process runs for a (very) long time, a  
variable will still be referenced by the parent process even though  
the child -async process has long since died, it's only when both the  
parent and child processes have ended that the variables will be  
garbage collected.

Anyway, thank you Fletcher (as ever) for the insights.

All the best

Simon Forster
_______________________________________________________
  LDML Ltd, 62 Pall Mall, London, SW1Y 5HZ, UK
  Tel: +44 (0)20 8133 0528   Fax: +44 (0)70 9230 5247
_______________________________________________________



------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage