Thread-objects, globals, and concurrency

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Thread-objects, globals, and concurrency

Steve Upton

thanks for all the discussion on the new world order concerning globals.

But now I need to know what to do.

The guts (skin?) of our UI (field labels, localized field values, etc) is currently served from a global map. It's a great function and, in theory, should be a very efficient way of serving localized text right from RAM.

But in L9 (and I guess in L8, it turns out) we're going to run into serious concurrency issues.

I just can't bring myself to turn 1-5 DB hits into 10x that by using the DB for localized text.

How do we create concurrently accessible map structures for threading goodness? memcached?

Regards,

Steve Upton


#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Thread-objects, globals, and concurrency

stevepiercy
FYI, we have an issue in the Knop Project to track this concept.
https://github.com/knop-project/knop/issues/43

Ke mentioned there is a better implementation.

Anyway, after you set your globals in 8, do you write to them
again?  If access is write once/read many times, then the
implementation would be different than a model where there are
many writes and many reads.

memcached is one strategy of many options.

--steve


On 6/19/12 at 6:14 PM, [hidden email] (Steve Upton) pronounced:

>thanks for all the discussion on the new world order concerning globals.
>
>But now I need to know what to do.
>
>The guts (skin?) of our UI (field labels, localized field
>values, etc) is currently served from a global map. It's a
>great function and, in theory, should be a very efficient way
>of serving localized text right from RAM.
>
>But in L9 (and I guess in L8, it turns out) we're going to run
>into serious concurrency issues.
>
>I just can't bring myself to turn 1-5 DB hits into 10x that by
>using the DB for localized text.
>
>How do we create concurrently accessible map structures for
>threading goodness? memcached?
>
>Regards,
>
>Steve Upton
>
>
>#############################################################
>This message is sent to you because you are subscribed to
>the mailing list Lasso
>[hidden email]
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
-- --
Steve Piercy               Web Site Builder              
Soquel, CA
<[hidden email]>                  <http://www.StevePiercy.com/>

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Thread-objects, globals, and concurrency

Steve Upton

On Jun 19, 2012, at 9:38 PM, Steve Piercy - Web Site Builder wrote:

> FYI, we have an issue in the Knop Project to track this concept.
> https://github.com/knop-project/knop/issues/43
>
> Ke mentioned there is a better implementation.
>

I thought that Ke's implementation (mentioned recently here in a thread) suffered from the same issues.

> Anyway, after you set your globals in 8, do you write to them again?  If access is write once/read many times, then the implementation would be different than a model where there are many writes and many reads.
>

It's write once-in-a-while, read many (we clear & reload them every few weeks or so)

> memcached is one strategy of many options.

ok… has anyone used it for such purposes?

regards,

Steve


#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Thread-objects, globals, and concurrency

stevepiercy
On 6/19/12 at 10:40 PM, [hidden email] (Steve Upton) pronounced:

>On Jun 19, 2012, at 9:38 PM, Steve Piercy - Web Site Builder wrote:
>
>>FYI, we have an issue in the Knop Project to track this concept.
>>https://github.com/knop-project/knop/issues/43
>>
>>Ke mentioned there is a better implementation.
>>
>
>I thought that Ke's implementation (mentioned recently here in
>a thread) suffered from the same issues.

I don't have details.  I hope Ke can elaborate.

>>Anyway, after you set your globals in 8, do you write to them again?  If access
>is write once/read many times, then the implementation would be
>different than a model where there are many writes and many reads.
>>
>
>It's write once-in-a-while, read many (we clear & reload them
>every few weeks or so)

memcached will probably be one of your better options.

cache methods may be acceptable.
http://www.lassosoft.com/lassoDocs/languageReference/obj/cache

It looks like Lasso 9 uses threads to cache data, which should
be pretty fast.  See the method definition in the following file:
http://download.lassosoft.com/svn/lasso/lasso9_source/trunk/compat/compat.cache.lasso
that starts with:
define cache_data_server => thread {

>>memcached is one strategy of many options.
>
>ok… has anyone used it for such purposes?

Brad Lindsay for one.
https://github.com/bfad/Lasso-Memcached-Connector

Jono mentioned at LDC using it for something.

--steve


>regards,
>
>Steve
>
>
>#############################################################
>This message is sent to you because you are subscribed to
>the mailing list Lasso
>[hidden email]
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
-- --
Steve Piercy               Web Site Builder              
Soquel, CA
<[hidden email]>                  <http://www.StevePiercy.com/>

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Thread-objects, globals, and concurrency

Ke Carlton-3
In reply to this post by Steve Upton
It's worth pointing out that providing your not under serious load, thread objects are up to the task - especially if used appropriately.

Just ensure you have a separate thread object per thing. ie. One for your menu, one for your settings etc.

Also, it's better to request an answer from a thread object as opposed to returning say an entire map - copying data is slow, so it's better to do something like thread_object->find('x') as opposed to thread_object->settingsmap->find('x'). The first example would likely take a few hundred micros, whilst the second may take a few milliseconds depending on the size of the map (going off memory).

The example I sent through was an extreme to highlight what goes on. If on a typical page request all of your calls to thread objects take 10 - 20 milliseconds, then your unlikely to notice thread objects locking until your hitting 50 requests a second or so...

That said; Each application is different, one of the most productive things you can do in terms of performance is first acquire a clear understanding of what's happening when and how long it's taking - use something like www.l-debug.org to do so...

From there you be able to test and compare approaches and use the one that works best for your application.

Ke - sent from a phone

On 20 Jun 2012, at 06:40, Steve Upton <[hidden email]> wrote:

> thought that Ke's implementation (mentioned recently here in a thread) suffered from the same issues
#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Thread-objects, globals, and concurrency

Brad Lindsay-2
In reply to this post by stevepiercy
On Jun 20, 2012, at 5:50 AM, Steve Piercy - Web Site Builder <[hidden email]> wrote:

>>> memcached is one strategy of many options.
>>
>> ok… has anyone used it for such purposes?
>
> Brad Lindsay for one.
> https://github.com/bfad/Lasso-Memcached-Connector

A couple of quick notes:

1) I haven't actually used memcached in production, just created that connector. However, it's very straightforward to setup and use.

2) At LDC, LassoSoft announced that they will have an official memcached connector that they will be supporting, so if you can wait for that to come out, it might be better than using my connector. Hopefully that'll be out soon.


Brad

#############################################################
This message is sent to you because you are subscribed to
  the mailing list Lasso
[hidden email]
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>