Lasso Scheduled Events

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Lasso Scheduled Events

Brian Loomis-3
I'm working on a system that is under extremely high load during a registration process, where about 400 users an hour are attempting to register.

One of our site bottlenecks are Lasso events that are triggered in an event to make FileMaker writebacks (from a MySQL based site) happen after users have updated certain selections.

There writebacks comprise 80 - 100 FIleMaker lasso calls, all on tuned layouts with only the fields used and each one returns 400ms - 800ms times.  In total the writebacks cause Lasso to take 50 - 70 seconds to execute the scheduled event, across multiple users is causing Lasso to crash every 40 minutes or so after stopping completely for ten minutes.

In an effort to make this scale better I'm recommending they fire up another VM with Lasso for the writebacks and leave the production site for the registrations in MySQL and the minimal FM calls for value lists etc..

The documentation for event schedule is as follows:

http://reference.lassosoft.com/Reference.LassoApp?lassoscript/browse

"This tag can be used to execute format files on the local server or to trigger CGIs on remote server"

Does anyone have any experience doing this or think that breaking off these processes to another Lasso VM this way would be effective?

BTW this is windows and IIS but that's probably not pertinent to what's happening.

Brian Loomis
http://www.virtualrelations.us
(208) 639-2569 - 208 NEW BLOX
-- email checked daily --


--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Lasso Scheduled Events

stevepiercy
The FM WPE is slow.  Assuming the only reason that the client
requires FM as a datasource is for FMP as a desktop client, can
you use ESS to get data from MySQL instead?  Then you would not
need to write data back to FM: it would reside in MySQL.  I'm
doing that myself with two clients.

Lasso events won't solve the problem of writing a large amount
of data or lots of requests to FM through the slow WPE.  Your
script needs to throttle the rate of writes to FM somehow--maybe
with [sleep] in a loop, or some test that indicates the write
has completed.

--steve


On 5/31/10 at 7:39 PM, [hidden email] (Brian Loomis) pronounced:

>I'm working on a system that is under extremely high load
>during a registration process, where about 400 users an hour
>are attempting to register.
>
>One of our site bottlenecks are Lasso events that are triggered
>in an event to make FileMaker writebacks (from a MySQL based
>site) happen after users have updated certain selections.
>
>There writebacks comprise 80 - 100 FIleMaker lasso calls, all
>on tuned layouts with only the fields used and each one returns
>400ms - 800ms times.  In total the writebacks cause Lasso to
>take 50 - 70 seconds to execute the scheduled event, across
>multiple users is causing Lasso to crash every 40 minutes or so
>after stopping completely for ten minutes.
>
>In an effort to make this scale better I'm recommending they
>fire up another VM with Lasso for the writebacks and leave the
>production site for the registrations in MySQL and the minimal
>FM calls for value lists etc..
>
>The documentation for event schedule is as follows:
>
>http://reference.lassosoft.com/Reference.LassoApp?lassoscript/browse
>
>"This tag can be used to execute format files on the local
>server or to trigger CGIs on remote server"
>
>Does anyone have any experience doing this or think that
>breaking off these processes to another Lasso VM this way would
>be effective?
>
>BTW this is windows and IIS but that's probably not pertinent to what's happening.
>
>Brian Loomis
>http://www.virtualrelations.us
>(208) 639-2569 - 208 NEW BLOX
>-- email checked daily --
>
>
>--
>This list is a free service of LassoSoft: http://www.LassoSoft.com/
>Search the list archives: http://www.ListSearch.com/Lasso/Browse/
>Manage your subscription: http://www.ListSearch.com/Lasso/
>
>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
-- --
Steve Piercy               Web Site Builder              
Soquel, CA
<[hidden email]>                  <http://www.StevePiercy.com/>


--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Lasso Scheduled Events

Brian Loomis-3
Didn't you mean SLOW?

On May 31, 2010, at 10:07 PM, Steve Piercy - Web Site Builder wrote:

> The FM WPE is slow

Brian Loomis
http://www.virtualrelations.us
(208) 639-2569 - 208 NEW BLOX
-- email checked daily --


--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Lasso Scheduled Events

stevepiercy
If the WPE was any slower, it would be in reverse.

--steve


On 5/31/10 at 10:22 PM, [hidden email] (Brian Loomis) pronounced:

> Didn't you mean SLOW?
>
> On May 31, 2010, at 10:07 PM, Steve Piercy - Web Site Builder wrote:
>
> > The FM WPE is slow
>
> Brian Loomis
> http://www.virtualrelations.us
> (208) 639-2569 - 208 NEW BLOX
> -- email checked daily --
>
>
> --
> This list is a free service of LassoSoft: http://www.LassoSoft.com/
> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
> Manage your subscription: http://www.ListSearch.com/Lasso/
>
>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Steve Piercy               Web Site Builder               Soquel, CA
<[hidden email]>                  <http://www.StevePiercy.com/>


--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Lasso Scheduled Events

Jussi Hirvi-2
Funny. I still have an old FMP 4 running as a web db for some clients,
and I see no need to "upgrade" FMP to a newer version. Eespecially as
the regular FMP 4 client licence is valid for web use.

A newer version of FMP on the same hw would probably only be slower.

A good little argument against technological determinism and the myth of
"everything gets better". Small/old is sometimes beautiful.

- Jussi

On 1.6.2010 7.51, Steve Piercy - Web Site Builder wrote:

> If the WPE was any slower, it would be in reverse.
>
> --steve
>
>
> On 5/31/10 at 10:22 PM, [hidden email] (Brian Loomis) pronounced:
>
>> Didn't you mean SLOW?
>>
>> On May 31, 2010, at 10:07 PM, Steve Piercy - Web Site Builder wrote:
>>
>>> The FM WPE is slow
>>
>> Brian Loomis
>> http://www.virtualrelations.us
>> (208) 639-2569 - 208 NEW BLOX
>> -- email checked daily --
>>
>>
>> --
>> This list is a free service of LassoSoft: http://www.LassoSoft.com/
>> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
>> Manage your subscription: http://www.ListSearch.com/Lasso/
>>
>>
>
> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> Steve Piercy               Web Site Builder               Soquel, CA
> <[hidden email]>                   <http://www.StevePiercy.com/>
>
>
> --
> This list is a free service of LassoSoft: http://www.LassoSoft.com/
> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
> Manage your subscription: http://www.ListSearch.com/Lasso/
>
>
>


--
Jussi Hirvi * Green Spot
Topeliuksenkatu 15 C * 00250 Helsinki * Finland
Tel. +358 9 493 981 * Mobile +358 40 771 2098 (only sms)
[hidden email] * http://www.greenspot.fi

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Lasso Scheduled Events

Marc Vos
In reply to this post by Brian Loomis-3
>
> BTW this is windows and IIS but that's probably not pertinent to  
> what's happening.

Well, don't say that. I have a Lasso 8.5.6 running on a virtual  
Windows 2003 server with IIS, which connects via JDBC to other virtual  
servers every two minutes and it crashes after running 2 days. I also  
tried a scheduled event to restart Lasso at a certain time to prevent  
it from crashing, but then Lasso can't get JDBC to connect without a  
manual kick in the butt.

So I am moving to CentOS because I think memory management is better  
there.

- -
Marc

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Lasso Scheduled Events

Brian Loomis-3
Yeah I just spent the last 2 hours moving the site over to a Centos 5.4 VM running on Parallels on OS X Server on a dual 2.0 dual core xServe (4 CPU).  On a VM with 1Gb Ram and 1 CPU is twice as fast as 2 CPU 4Gb RAM IIS machine.

On Jun 1, 2010, at 1:29 AM, Marc Vos wrote:

> So I am moving to CentOS because I think memory management is better there.

Brian Loomis
http://www.virtualrelations.us
(208) 639-2569 - 208 NEW BLOX
-- email checked daily --


--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Lasso Scheduled Events

Johan Solve
In reply to this post by Brian Loomis-3
You don't need a separate VM for that.

Just store the live registrations in MySQL, and write a script to move
registrations from MySQL to FM in batches maybe once every 10 minutes
or so. You could run that in a separate LassoSite process to isolate
it from production, but the script won't run hot. Just lock it up so
it doesn't run in parallell.


On Tue, Jun 1, 2010 at 3:39 AM, Brian Loomis <[hidden email]> wrote:

> I'm working on a system that is under extremely high load during a registration process, where about 400 users an hour are attempting to register.
>
> One of our site bottlenecks are Lasso events that are triggered in an event to make FileMaker writebacks (from a MySQL based site) happen after users have updated certain selections.
>
> There writebacks comprise 80 - 100 FIleMaker lasso calls, all on tuned layouts with only the fields used and each one returns 400ms - 800ms times.  In total the writebacks cause Lasso to take 50 - 70 seconds to execute the scheduled event, across multiple users is causing Lasso to crash every 40 minutes or so after stopping completely for ten minutes.
>
> In an effort to make this scale better I'm recommending they fire up another VM with Lasso for the writebacks and leave the production site for the registrations in MySQL and the minimal FM calls for value lists etc..
>
> The documentation for event schedule is as follows:
>
> http://reference.lassosoft.com/Reference.LassoApp?lassoscript/browse
>
> "This tag can be used to execute format files on the local server or to trigger CGIs on remote server"
>
> Does anyone have any experience doing this or think that breaking off these processes to another Lasso VM this way would be effective?
>
> BTW this is windows and IIS but that's probably not pertinent to what's happening.
>
> Brian Loomis
> http://www.virtualrelations.us
> (208) 639-2569 - 208 NEW BLOX
> -- email checked daily --
>
>
> --
> This list is a free service of LassoSoft: http://www.LassoSoft.com/
> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
> Manage your subscription: http://www.ListSearch.com/Lasso/
>
>
>



--
Mvh
Johan Sölve
____________________________________
Montania System AB
Halmstad   Stockholm   Malmö
http://www.montania.se

Johan Sölve
Mobil +46 709-51 55 70
[hidden email]

Kristinebergsvägen 17, S-302 41 Halmstad, Sweden
Telefon +46 35-136800 |  Fax +46 35-136801

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Lasso Scheduled Events

maxwellk2
In reply to this post by Brian Loomis-3
FileMaker's ESS would be a good alternative, but ever since ESS in FM9 I've been neglecting FileMaker's old 'Import Records' script step with an ODBC SQL query as the source. If you're just importing data from SQL to FileMaker in one direction I would highly suggest this method. I had to try this recently because ESS was just way too slow, especially when mimicking SQL joins. I found the ODBC import to be magnitudes faster than ESS - 500,000+ records and 15 fields (half are joins) in just under 10 minutes.

You can also just import the new record's key fields (via ODBC or ESS) then use ESS to relate to the rest of the fields dynamically in your SQL source if the data doesn't absolutely have to live in FileMaker.

If you're running FileMaker Server 10 or 11 I'm pretty sure (as yet untested) you can automated the ODBC import with a FileMaker Server scheduled script at whatever interval makes sense, as long as the Server is configured with the same ODBC connection as on your desktop when creating the script. However, I've had problems with Server scripted ESS imports.

Anything to avoid using FileMaker's WPE is going to be faster.

BTW, FileMaker publicly announced "...the XSLT API will not be included in the new web publishing architecture in the next version of FileMaker software..." - good riddance. I was just down at FileMaker, Inc. and they promised me it was so they could focus on improving their XML API for the WPE.

-Max


On May 31, 2010, at 6:39 PM, Brian Loomis wrote:

> I'm working on a system that is under extremely high load during a registration process, where about 400 users an hour are attempting to register.
>
> One of our site bottlenecks are Lasso events that are triggered in an event to make FileMaker writebacks (from a MySQL based site) happen after users have updated certain selections.
>
> There writebacks comprise 80 - 100 FIleMaker lasso calls, all on tuned layouts with only the fields used and each one returns 400ms - 800ms times.  In total the writebacks cause Lasso to take 50 - 70 seconds to execute the scheduled event, across multiple users is causing Lasso to crash every 40 minutes or so after stopping completely for ten minutes.
>
> In an effort to make this scale better I'm recommending they fire up another VM with Lasso for the writebacks and leave the production site for the registrations in MySQL and the minimal FM calls for value lists etc..
>
> The documentation for event schedule is as follows:
>
> http://reference.lassosoft.com/Reference.LassoApp?lassoscript/browse
>
> "This tag can be used to execute format files on the local server or to trigger CGIs on remote server"
>
> Does anyone have any experience doing this or think that breaking off these processes to another Lasso VM this way would be effective?
>
> BTW this is windows and IIS but that's probably not pertinent to what's happening.
>
> Brian Loomis
> http://www.virtualrelations.us
> (208) 639-2569 - 208 NEW BLOX
> -- email checked daily --

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/