Getting info in and out of asasync

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Getting info in and out of asasync

Ari Denison
I'm using Lasso 9.2.5 on CentOS 6.4

I need a reliable way to get parameters from the main script to an asynchronously processed block of code.

I am trying to loop through a large list of PDFs so that I can use dmtxread to get information from a datamatrix-barcode coversheet on each PDF.

I am using this basic format:

{

        shell('/usr/bin/convert -scale 25% -quality 100 -density 576 /longpath/source/13-02386osc.pdf[0] /longpath/png/13-02386osc.png')

        shell('/usr/bin/dmtxread --page=1 -n --square-deviation=30 --threshold=15 /longpath/png/13-02386osc.png')

}->asasync();

Which does seem to spawn a new thread and doesn't delay the main script, but I can not, FOR THE LIFE OF ME, figure out how to get information into this asasync block. Globals are not an option for my environment.

I have tried:

}
        // do stuff
}->asasync(-params(array('PDFFileName'='13-02386osc.pdf', 'PDFPath'='/long/path/')))

I have also tried Bil's method (from: http://www.lassosoft.com/Spawning-Asynchronous-Threads):

}
        Lasso_ExecutionTimeLimit(60 * 60); // in seconds, set this to something reasonable, currently set to 1 hour
    Thread_SetPriority(Thread_GetCurrentID, Thread_Priority_Low); // set priority to low, give more priority to normal web requests (can also make it high if needed)

    // recreate vars - don't change the code below - this makes all the vars on the page available to the async thread
   
    iterate(params->find('-vars')->get(1)->value, local('var')) => {^
    var(#var->name=#var->value)
    ^}
}->asasync(-params=array( '-vars' = vars))

Both of these options give me an error similar to this in Lasso 9.5.2 on CentOS 6.4
Error Msg: Definition Not Found: capture->asasync(-params=array) Candidates were: capture->asasync()

I have even tried Bil's suggestion of using include_url(-postparms(), -nodata) as an endpoint api but that doesn't seem to spawn an independent thread at all in Lasso 9. I just sit and wait and wait as though the processing was done from the main script.

Any suggestions are VERY welcome.
Thanks!
Ari



#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Getting info in and out of asasync

Jolle Carlestam-3
10 jul 2013 kl. 08:31 skrev Ari Denison <[hidden email]>:

> I'm using Lasso 9.2.5 on CentOS 6.4
>
> I need a reliable way to get parameters from the main script to an asynchronously processed block of code.
>
> I am trying to loop through a large list of PDFs so that I can use dmtxread to get information from a datamatrix-barcode coversheet on each PDF.
>
> I am using this basic format:
>
> {
>
> shell('/usr/bin/convert -scale 25% -quality 100 -density 576 /longpath/source/13-02386osc.pdf[0] /longpath/png/13-02386osc.png')
>
> shell('/usr/bin/dmtxread --page=1 -n --square-deviation=30 --threshold=15 /longpath/png/13-02386osc.png')
>
> }->asasync();
>
> Which does seem to spawn a new thread and doesn't delay the main script, but I can not, FOR THE LIFE OF ME, figure out how to get information into this asasync block. Globals are not an option for my environment.

asasync is only a compat method for splitThread. I suggest using splitThread directly instead.

To start getting grips of what you can do with splitThread take a look here:
http://www.lassosoft.com/lassoDocs/languageReference/obj/capture/splitthread

It will not answer your question but does contain some good info. Like: "new thread will contain copies of the local variables that are active at the time the new thread is created"

If you want to send to and receive info from a split thread there's some info on how to do that here:
http://lassoguide.com/language/threading.html#splitting-threads

HDB
Jolle

#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Getting info in and out of asasync

Ke Carlton-3
The content on Lassoguide.com is looking really useful — well written and
clearly presented (not so sure about the interface, although it does look
young).

On 10 July 2013 09:00, Jolle Carlestam <[hidden email]> wrote:

>
> If you want to send to and receive info from a split thread there's some
> info on how to do that here:
> http://lassoguide.com/language/threading.html#splitting-threads
>
> HDB
> Jolle
>
>

#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Getting info in and out of asasync

Jolle Carlestam-3
10 jul 2013 kl. 12:04 skrev Ke Carlton <[hidden email]>:

> The content on Lassoguide.com is looking really useful — well written and
> clearly presented

I second that. Quickly becoming a prime source of knowledge for all things lasso 9 related.

HDB
Jolle


#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Getting info in and out of asasync

Ari Denison
In reply to this post by Jolle Carlestam-3
I'm making progress. I made an attempt with split_thread but I found that:

1. They do not execute anything having to do with os_process, "shell" in particular in this case. I tried a number of options, whoami being the most simple.

2. They do NOT have access to variables available at the time of the thread creation.

3. They do seem to execute asynchronously.

So. I'm still stuck where I was. I see that there's a way to communicate with split threads, but if I can't use os_process inside a split thread it doesn't really matter.  

Any other options?

See example below.

        var(foo='123')
        stdoutnl('---------- START MAIN SCRIPT ' + date + ': ' + var('foo') + '----------')
       
        split_thread => {
               
                sleep(3000) // used to see if this slows down the main script load time. It doesn't. Great.
               
                // The variable "foo" is NOT available inside the thread.
                stdoutnl('---------- TEST THREAD-PART-A ' + date + ':' + var('foo') + '----------')
               
                define shell(cmd::string) => {
                    local(os = lasso_version(-lassoplatform))
                    local(this = file_forceRoot(response_Path))
                    #this->replace('//','/')
                    if(#os >> 'Win') => {
                        local(shell = os_process('cmd',(:'/c cd ' + #this + ' && ' + #cmd)))        
                    else(#os >> 'Mac')
                        local(shell = os_process('/bin/bash', (:'-c','cd '+#this+'; '+#cmd)))
                    else
                        //local(shell = os_process('/bin/sh', (:'cd '+#this+'; '+#cmd)))
                        local(shell = os_process('/bin/sh', (:'-c','cd '+#this+'; '+#cmd)))
                    }
                    local(out = #shell->readString)
                    !#out->size ? #out = #shell->readerror
                    #shell->close
                    return(#out)
            }
           
            // This line will print.
                stdoutnl('---------- TEST 6 THREAD-PART-B: ' + date + '----------')
               
                local('whoIam'=shell('/usr/bin/whoami'))
               
                // This line will not print. Shell/os_process caused a problem.
                stdoutnl('---------- TEST 6 THREAD-PART-B: ' + date + ': ' + local('whoIam') + '----------')
               
        }
        // This line prints.
        stdoutnl('---------- END MAIN SCRIPT ' + date + ': ' + var('foo') + '----------')





On Jul 10, 2013, at 3:00 AM, Jolle Carlestam <[hidden email]> wrote:

10 jul 2013 kl. 08:31 skrev Ari Denison <[hidden email]>:

> I'm using Lasso 9.2.5 on CentOS 6.4
>
> I need a reliable way to get parameters from the main script to an asynchronously processed block of code.
>
> I am trying to loop through a large list of PDFs so that I can use dmtxread to get information from a datamatrix-barcode coversheet on each PDF.
>
> I am using this basic format:
>
> {
>
> shell('/usr/bin/convert -scale 25% -quality 100 -density 576 /longpath/source/13-02386osc.pdf[0] /longpath/png/13-02386osc.png')
>
> shell('/usr/bin/dmtxread --page=1 -n --square-deviation=30 --threshold=15 /longpath/png/13-02386osc.png')
>
> }->asasync();
>
> Which does seem to spawn a new thread and doesn't delay the main script, but I can not, FOR THE LIFE OF ME, figure out how to get information into this asasync block. Globals are not an option for my environment.

asasync is only a compat method for splitThread. I suggest using splitThread directly instead.

To start getting grips of what you can do with splitThread take a look here:
http://www.lassosoft.com/lassoDocs/languageReference/obj/capture/splitthread

It will not answer your question but does contain some good info. Like: "new thread will contain copies of the local variables that are active at the time the new thread is created"

If you want to send to and receive info from a split thread there's some info on how to do that here:
http://lassoguide.com/language/threading.html#splitting-threads

HDB
Jolle

#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
 the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>


#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Getting info in and out of asasync

Brad Lindsay
On 7/10/13, 1:34 PM, Ari Denison wrote:
> I'm making progress. I made an attempt with split_thread but I found that:
>
> 1. They do not execute anything having to do with os_process, "shell" in particular in this case. I tried a number of options, whoami being the most simple.

Have you tried using sys_process?
http://lassoguide.com/operations/sys_process.html


> 2. They do NOT have access to variables available at the time of the thread creation.

They don't have access to thread variables (those created with [var]),
but they do with local variables (those created with [local]).

Brad

#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Getting info in and out of asasync

Jolle Carlestam-3
In reply to this post by Ari Denison
10 jul 2013 kl. 19:34 skrev Ari Denison <[hidden email]>:

> 1. They do not execute anything having to do with os_process, "shell" in particular in this case. I tried a number of options, whoami being the most simple.
>
> 2. They do NOT have access to variables available at the time of the thread creation.

Here's a complete example contradicting these statements:

<?LassoScript

local(
        path = '/var/lasso/instances/',
        check = 'first'
)

local(code = {
        local(process = sys_process('/bin/ls', (:#path)))
        #process -> wait
        stdoutnl(#check)
        stdoutnl(#process -> readerror)
        stdoutnl(#process -> read)
        #process -> close
})

#code() // this is executed as part of the original thread.

#path = '/var/lasso/'
#check = 'second'
split_thread => #code // here we run the code again, this time with new values for the locals and in a new thread

#path = '/var/lasso/home/'
#check = 'third'
split_thread => #code // and the same code a last time. Also in a new thread

?>

Here's the result in the console:
first

default

second

global_cli_home
home
instances
lasso.err.txt
lasso.out.txt

third

JDBCDrivers
JavaLibraries
LassoApps
LassoLibraries
LassoModules
SQLiteDBs
lasso9.im.fastcgi.sock


So, it is possible to do sys_process calls in an async thread.
Locals declared at the point of triggering the new thread are available inside the thread. Note; LOCALS not VARS.

HDB
Jolle

#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Getting info in and out of asasync

Jolle Carlestam-3
10 jul 2013 kl. 23:29 skrev Jolle Carlestam <[hidden email]>:

Some comments on what's done here.

Sys_process is the preferred call in Lasso 9. Os_process is still supported but can be avoided.

> <?LassoScript
>
> local(
> path = '/var/lasso/instances/',

In order for sys_process to work, all paths should be from root of the server.

> check = 'first'
> )
>
> local(code = {

Putting code inside a code block, that is within curly brackets, will prepare the code but not run it. The code in the code block is not executed until it is invoked.

> local(process = sys_process('/bin/ls', (:#path)))

When calling commands for sysprocess the commands have to be given with a full path from server root. When running from command line it is often sufficient to just write the command because the shell takes care of finding the path to the command for you. But sys_process does not execute within a shell so that help is not available. Note that some libraries have different locations depending on type of server. At least I've found that to be the case with for example the ImageMagick install. So if you move between OS X and Linux, like I do, make sure to enter a path relevant for the OS in question.

> #process -> wait

This is a usual miss. If you don't allow the Lasso process to wait for the OS call to finish you'll get unpredictable results. Sometimes it will work anyway, other times not.

> stdoutnl(#check)
> stdoutnl(#process -> readerror)

This will, in this case, return nothing since there's no error. But if you try sys_process and don't get the result you expect, try checking the readerror result for clues. If you don't care calling readerror can be omitted.

> stdoutnl(#process -> read)
> #process -> close

It is good manner to always close the sys_process call when done.

> })
>
> #code() // this is executed as part of the original thread.

The above is short hand for => invoke
In other words to run the code found inside the code block.

>
> #path = '/var/lasso/'
> #check = 'second'
> split_thread => #code // here we run the code again, this time with new values for the locals and in a new thread

The code in the local #code is now sent of to a new thread for execution. With the local variables sent along holding the values as they look at this point.

>
> #path = '/var/lasso/home/'
> #check = 'third'
> split_thread => #code // and the same code a last time. Also in a new thread
>
> ?>

That's it.

HDB
Jolle

#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Getting info in and out of asasync

Ari Denison
Thanks for the detailed explanation. I'll see what I can do with this and report back to the list.

On Jul 10, 2013, at 5:01 PM, Jolle Carlestam <[hidden email]> wrote:

10 jul 2013 kl. 23:29 skrev Jolle Carlestam <[hidden email]>:

Some comments on what's done here.

Sys_process is the preferred call in Lasso 9. Os_process is still supported but can be avoided.

> <?LassoScript
>
> local(
> path = '/var/lasso/instances/',

In order for sys_process to work, all paths should be from root of the server.

> check = 'first'
> )
>
> local(code = {

Putting code inside a code block, that is within curly brackets, will prepare the code but not run it. The code in the code block is not executed until it is invoked.

> local(process = sys_process('/bin/ls', (:#path)))

When calling commands for sysprocess the commands have to be given with a full path from server root. When running from command line it is often sufficient to just write the command because the shell takes care of finding the path to the command for you. But sys_process does not execute within a shell so that help is not available. Note that some libraries have different locations depending on type of server. At least I've found that to be the case with for example the ImageMagick install. So if you move between OS X and Linux, like I do, make sure to enter a path relevant for the OS in question.

> #process -> wait

This is a usual miss. If you don't allow the Lasso process to wait for the OS call to finish you'll get unpredictable results. Sometimes it will work anyway, other times not.

> stdoutnl(#check)
> stdoutnl(#process -> readerror)

This will, in this case, return nothing since there's no error. But if you try sys_process and don't get the result you expect, try checking the readerror result for clues. If you don't care calling readerror can be omitted.

> stdoutnl(#process -> read)
> #process -> close

It is good manner to always close the sys_process call when done.

> })
>
> #code() // this is executed as part of the original thread.

The above is short hand for => invoke
In other words to run the code found inside the code block.

>
> #path = '/var/lasso/'
> #check = 'second'
> split_thread => #code // here we run the code again, this time with new values for the locals and in a new thread

The code in the local #code is now sent of to a new thread for execution. With the local variables sent along holding the values as they look at this point.

>
> #path = '/var/lasso/home/'
> #check = 'third'
> split_thread => #code // and the same code a last time. Also in a new thread
>
> ?>

That's it.

HDB
Jolle

#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
 the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>


#############################################################

Attend the Lasso Developer Conference 2013!
Sept 12-14, 2013 in Niagara Falls, Canada
http://www.lassosoft.com/LDC-niagara-falls-2013

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>