Concurrent Read Access to Thread Object that Emulates Map

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Concurrent Read Access to Thread Object that Emulates Map

Justin Dennis-3
Hi -

Also on Stack Exchange to spread the good word:
http://stackoverflow.com/questions/27930539/concurrent-read-access-to-thread-object-that-emulates-map

I am experiencing (very) slow page load times that increase proportionately
to the number of active users on the system. I have a hunch that this is
related to a custom defined thread object:

    define stageStoreCache => thread {
    parent map
    public oncreate() => ..oncreate()
    }

This stageStoreCache object simply mimics the behavior of a map whose data
available across the entire instance.

Many threads are reading it and very few threads are writing to it. Is this
a poorly conceived solution to having a large map of data available across
the instance? It's a fairly large map of maps that when exported to
map->asstring can exceed 5MB. The objective is to prevent translating data
stored as JSON in the database to Lasso types on the fly.

It seems that the large size of the stageStoreCache is not what causes
problems. It seems to really be the number of concurrent users on the
system.

Thanks for any insight you can offer.

-Justin

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Brad Lindsay
My reply can be found on StackOverflow

Brad

On January 13, 2015 at 3:03:52 PM, Justin Dennis ([hidden email]) wrote:

> Hi -
>
> Also on Stack Exchange to spread the good word:
> http://stackoverflow.com/questions/27930539/concurrent-read-access-to-thread-object-that-emulates-map 
>
> I am experiencing (very) slow page load times that increase proportionately
> to the number of active users on the system. I have a hunch that this is
> related to a custom defined thread object:
>
> define stageStoreCache => thread {
> parent map
> public oncreate() => ..oncreate()
> }
>
> This stageStoreCache object simply mimics the behavior of a map whose data
> available across the entire instance.
>
> Many threads are reading it and very few threads are writing to it. Is this
> a poorly conceived solution to having a large map of data available across
> the instance? It's a fairly large map of maps that when exported to
> map->asstring can exceed 5MB. The objective is to prevent translating data
> stored as JSON in the database to Lasso types on the fly.
>
> It seems that the large size of the stageStoreCache is not what causes
> problems. It seems to really be the number of concurrent users on the
> system.
>
> Thanks for any insight you can offer.
>
> -Justin
>
> #############################################################
>
> This message is sent to you because you are subscribed to
> the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to:
> Send administrative queries to
>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Ke Carlton-3
In reply to this post by Justin Dennis-3
Hello Justin,

Thread objects are single threaded — read or write. So you must keep that
in mind that when retrieving data from them.

Ideally, stageStoreCache->find should return snippets of information as the
data is returned via a copy — which is a slow operation. So for example
avoid doing something like:

stageStoreCache->insert(
'example' = map(
'with' = 'lots',
'of' = 'neat',
'settings' = array(1,2,3)
)
)

// This is a heavy operation
local(settings) = stageStoreCache->find('example')->find('settings')

// Instead do something like:
local(settings) = stageStoreCache->findsetting('example','settings')

Ensure you're only retrieving the items once per thread with something like
the below:

define user_settings => var(user_settings) || $user_settings
:= stageStoreCache->find('user_settings')


Also, consider using multiple threads for different types of data, for
example:

userStore->find('xxxx')
appStore->find('xxxx')
Settings->find('xxxx')


Finally is a thread object the best thing to use here? something like
memcached may be more suited depending on the type of data (especially if
something like rendered html). See:

https://github.com/bfad/Lasso-Memcached-Connector
https://github.com/LassoSoft/Memcached


Ke



On 14 January 2015 at 09:03, Justin Dennis <[hidden email]> wrote:

> Hi -
>
> Also on Stack Exchange to spread the good word:
>
> http://stackoverflow.com/questions/27930539/concurrent-read-access-to-thread-object-that-emulates-map
>
> I am experiencing (very) slow page load times that increase proportionately
> to the number of active users on the system. I have a hunch that this is
> related to a custom defined thread object:
>
>     define stageStoreCache => thread {
>     parent map
>     public oncreate() => ..oncreate()
>     }
>
> This stageStoreCache object simply mimics the behavior of a map whose data
> available across the entire instance.
>
> Many threads are reading it and very few threads are writing to it. Is this
> a poorly conceived solution to having a large map of data available across
> the instance? It's a fairly large map of maps that when exported to
> map->asstring can exceed 5MB. The objective is to prevent translating data
> stored as JSON in the database to Lasso types on the fly.
>
> It seems that the large size of the stageStoreCache is not what causes
> problems. It seems to really be the number of concurrent users on the
> system.
>
> Thanks for any insight you can offer.
>
> -Justin
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>   the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Jolle Carlestam-2
In reply to this post by Brad Lindsay
As can mine.

HDB
Jolle

13 jan 2015 kl. 22:51 skrev Brad Lindsay <[hidden email]>:

> My reply can be found on StackOverflow
>
> Brad
>
> On January 13, 2015 at 3:03:52 PM, Justin Dennis ([hidden email]) wrote:
>> Hi -
>>
>> Also on Stack Exchange to spread the good word:
>> http://stackoverflow.com/questions/27930539/concurrent-read-access-to-thread-object-that-emulates-map 


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Justin Dennis-3
Thank you all for your insights and suggestions.

I realize now that I have pretty much violated every best practice you've
mentioned. I need to refactor the whole mess.

When it was just me or five users testing, it was infinitely faster to
cache like this. Now I realize that the number of copies of 5 MB of data
(after the cache is "warmed up") has increased many times as utilization of
this thread object has spread throughout our apps with many more concurrent
users.

I will follow-up after I've made some headway. Thank you all again.

- Justin

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Justin Dennis-3
In reply to this post by Justin Dennis-3
Hi All -

An update: I reviewed all use of stageStoreCache. All ->find operations are
just one level deep. I put a timer on the ->find operations and found that
the vast majority of times it takes between 0 and 1ms (sometimes 2ms). But
other times it takes as many as 30000 ms (30 seconds!) or more. I am taking
_date_msec snapshots one line before and one line after and logging to
database.

I am beginning to think that something else is causing the initial
slowness, which then cascades until the backlog of accesses catches up?

Here's a sample from my timer log:

msecTimestamp msec114212636342731142126363972011421263662996114212636632041
1421263707032214212637137662142126371978813219142126374558268641421263745583
4939142126374558412291421263745586

It's humming right along until it hits a brick wall and takes 13 seconds to
return. A few lines later and it's back to 1ms return times. These are all
called from the same page in the same way (seeking different keys of
course). I cannot duplicate based on the key. It really seems to be
affected by some outside influence.

The stageStoreCache map is the exact same across this sample. Of course
there's other activity on the server, and I have no idea how to find what's
causing it to choke.

Any ideas or pointers as to how I can test or troubleshoot this issue are
greatly appreciated.

- Justin


On Tue, Jan 13, 2015 at 3:03 PM, Justin Dennis <[hidden email]> wrote:

> Hi -
>
> Also on Stack Exchange to spread the good word:
>
> http://stackoverflow.com/questions/27930539/concurrent-read-access-to-thread-object-that-emulates-map
>
> I am experiencing (very) slow page load times that increase
> proportionately to the number of active users on the system. I have a hunch
> that this is related to a custom defined thread object:
>
>     define stageStoreCache => thread {
>     parent map
>     public oncreate() => ..oncreate()
>     }
>
> This stageStoreCache object simply mimics the behavior of a map whose data
> available across the entire instance.
>
> Many threads are reading it and very few threads are writing to it. Is
> this a poorly conceived solution to having a large map of data available
> across the instance? It's a fairly large map of maps that when exported to
> map->asstring can exceed 5MB. The objective is to prevent translating data
> stored as JSON in the database to Lasso types on the fly.
>
> It seems that the large size of the stageStoreCache is not what causes
> problems. It seems to really be the number of concurrent users on the
> system.
>
> Thanks for any insight you can offer.
>
> -Justin
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Jolle Carlestam-2
14 jan 2015 kl. 20:38 skrev Justin Dennis <[hidden email]>:

> An update: I reviewed all use of stageStoreCache. All ->find operations are
> just one level deep. I put a timer on the ->find operations and found that
> the vast majority of times it takes between 0 and 1ms (sometimes 2ms). But
> other times it takes as many as 30000 ms (30 seconds!) or more. I am taking
> _date_msec snapshots one line before and one line after and logging to
> database.
>
> I am beginning to think that something else is causing the initial
> slowness, which then cascades until the backlog of accesses catches up?

> It's humming right along until it hits a brick wall and takes 13 seconds to
> return. A few lines later and it's back to 1ms return times. These are all
> called from the same page in the same way (seeking different keys of
> course). I cannot duplicate based on the key. It really seems to be
> affected by some outside influence.

I’m going to repeat what’s been said about this question before to make sure that the insights of the complications are clear.
Each time you do
stageStoreCache -> find(’...')
you copy 5+ MB of data to the current thread. If you do that 10 times in one visit, to retrieve 10 different settings, that’s 50+ MB copied. On each call. Say you have 10 visitors side by side, that’s 500+ MB data shuffled. If you have 100 visitors it’s some 5 GB of data.

My guess is that the slowdowns you see are when Lasso hits memory limitations and have to start cleaning up and dumping what’s not actively used anymore. Or that it does some disk swapping in an effort to keep all data that you're requesting it to handle.

To have all that data in one huge map is a bad habit to begin with, it should be rewritten along the guidelines proposed to you.
In the meantime, you can go a long way by at least making sure that you only copy the data from the thread object once per visit. Here’s an way to do that:

define localStoreCache => var(__StageStoreCache) || $__StageStoreCache := stageStoreCache
// all data copied from stageStoreCache first time the method is called
// put this definition in the same place you have the definition for stageStoreCache

Replace any
stageStoreCache -> find(…)
with
localStoreCache -> find(...)

Like so:
localStoreCache -> find(’mysetting1’) // If this is the first call, the data is copied from the thread object
localStoreCache -> find(’mysetting2’) // no copying of data takes place

The above should be a fairly simple search and replace exercise until you can refactor your code to a more efficient handling.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Jolle Carlestam-2
15 jan 2015 kl. 10:53 skrev Jolle Carlestam <[hidden email]>:

> define localStoreCache => var(__StageStoreCache) || $__StageStoreCache := stageStoreCache

A moment of clarification. If there are readers that, like me, find that this looks like a very smart and efficient way of handling access to data, methods or types, they are right. But, I am not able to take credit for it. The idea is sprung from the mastermind of Ke Carlton. Thank him for choosing to use Lasso as his preferred programming language so that we can all ride on his shoulders and reach further into the sky.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Ke Carlton-3
In reply to this post by Justin Dennis-3
Hello Justin,

It doesn't sound like you've changed too much (still a single 5mb thread
object).

Properly stress testing it is a good idea:

stress.lasso
=========
local(keys) = (:'all','the','keys','hardcoded')

local(
    threads = 10,
    keysperpage = 30,
    thread_id = lasso_uniqueid
)

with i in 1 to #threads do {
    {
        sleep(math_random(0,10))
        local(keys) = (:'all','the','keys','hardcoded')

       stdoutnl(#thread_id +': ' + #i + ' = ' +
            (timer(1) => {
             with r in 1 to #keysperpage do {
                 stageStoreCache->find(
                    #keys->get(
                        math_random(1,#keys->size)
                    )
                )->size
             }
            })->last
        )

    }->asasync
}


Try calling stress.lasso once, then check the log. You should see something
like this:

8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 22 = (micros_average = 995.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 43 = (micros_average = 891.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 83 = (micros_average = 163.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 57 = (micros_average = 206.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 59 = (micros_average = 215.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 72 = (micros_average = 187.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 76 = (micros_average = 253.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 78 = (micros_average = 81.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 82 = (micros_average = 76.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 99 = (micros_average = 68.000000)
8A3AB528-BDAF-4CFD-8B81-08F8BD5835E3: 90 = (micros_average = 70.000000)

Which doesn't seem too bad — but the varied times are caused by read locks.

Then try something like this:

ab -c10 --n100 http://localhost/stress.lasso

43574970-E023-4608-B7F3-ACC3E8AF8A23: 42 = (micros_average = 1072455.000000)
73DFDF9D-790E-47E9-A856-8842D1CC248A: 75 = (micros_average = 7201.000000)
43574970-E023-4608-B7F3-ACC3E8AF8A23: 29 = (micros_average = 1072453.000000)
43574970-E023-4608-B7F3-ACC3E8AF8A23: 59 = (micros_average = 7299.000000)

Non-thread locking reads would be great — but they don't currently exist.

The solution depends entirely on how the store is being used and it may be
one of the previously mentioned approaches.

It's may be worth pointing out that using stdoutnl here will also form a
bottle neck here. Writing to a syslog server is a better option.

It is somewhat hard to suggest an exact solution as all systems are
different, but you should be able to find a sweet spot with a combination
what people have suggested.

Ke


On 15 January 2015 at 08:38, Justin Dennis <[hidden email]> wrote:

> Hi All -
>
> An update: I reviewed all use of stageStoreCache. All ->find operations are
> just one level deep. I put a timer on the ->find operations and found that
> the vast majority of times it takes between 0 and 1ms (sometimes 2ms). But
> other times it takes as many as 30000 ms (30 seconds!) or more. I am taking
> _date_msec snapshots one line before and one line after and logging to
> database.
>
> I am beginning to think that something else is causing the initial
> slowness, which then cascades until the backlog of accesses catches up?
>
> Here's a sample from my timer log:
>
> msecTimestamp msec114212636342731142126363972011421263662996114212636632041
>
> 1421263707032214212637137662142126371978813219142126374558268641421263745583
> 4939142126374558412291421263745586
>
> It's humming right along until it hits a brick wall and takes 13 seconds to
> return. A few lines later and it's back to 1ms return times. These are all
> called from the same page in the same way (seeking different keys of
> course). I cannot duplicate based on the key. It really seems to be
> affected by some outside influence.
>
> The stageStoreCache map is the exact same across this sample. Of course
> there's other activity on the server, and I have no idea how to find what's
> causing it to choke.
>
> Any ideas or pointers as to how I can test or troubleshoot this issue are
> greatly appreciated.
>
> - Justin
>
>
> On Tue, Jan 13, 2015 at 3:03 PM, Justin Dennis <[hidden email]> wrote:
>
> > Hi -
> >
> > Also on Stack Exchange to spread the good word:
> >
> >
> http://stackoverflow.com/questions/27930539/concurrent-read-access-to-thread-object-that-emulates-map
> >
> > I am experiencing (very) slow page load times that increase
> > proportionately to the number of active users on the system. I have a
> hunch
> > that this is related to a custom defined thread object:
> >
> >     define stageStoreCache => thread {
> >     parent map
> >     public oncreate() => ..oncreate()
> >     }
> >
> > This stageStoreCache object simply mimics the behavior of a map whose
> data
> > available across the entire instance.
> >
> > Many threads are reading it and very few threads are writing to it. Is
> > this a poorly conceived solution to having a large map of data available
> > across the instance? It's a fairly large map of maps that when exported
> to
> > map->asstring can exceed 5MB. The objective is to prevent translating
> data
> > stored as JSON in the database to Lasso types on the fly.
> >
> > It seems that the large size of the stageStoreCache is not what causes
> > problems. It seems to really be the number of concurrent users on the
> > system.
> >
> > Thanks for any insight you can offer.
> >
> > -Justin
> >
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>   the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Justin Dennis-3
Thank you all for your additional contributions. This is starting to make
sense and I am understanding now how badly I've done this.

I have some pages that call stageStoreCache 100 or more times. I originally
thought that just the result of the ->find was copied. But now I understand
that it *copies the whole thing* before the ->find returns... for *each*
->find.

Do I understand this correctly now? If so, then I think I am better
understanding your suggested solutions. Thank you again for your thoughtful
replies.

- Justin

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Brad Lindsay
I’m pretty sure Jolle’s assertion that “Each time you do stageStoreCache -> find(’...') you copy 5+ MB of data to the current thread” is wrong. Thread objects don’t produce a full copy of themselves each time you try to use them - they only copy the data to the thread object that you send and copy the data from the thread object that it sends back.

Calling [stageStoreCache->find('key')] copies the string object “key” to the thread object and then copies the return value from the “find” out to the current thread.

That being said, if those top-level objects stored are large, then that’s a large amount of data being copied. If what you need can be further drilled down to, well, see mine and Ke’s earlier suggestions.


Brad



On January 15, 2015 at 11:05:03 AM, Justin Dennis ([hidden email]) wrote:

> Thank you all for your additional contributions. This is starting to make
> sense and I am understanding now how badly I've done this.
>  
> I have some pages that call stageStoreCache 100 or more times. I originally
> thought that just the result of the ->find was copied. But now I understand
> that it *copies the whole thing* before the ->find returns... for *each*
> ->find.
>  
> Do I understand this correctly now? If so, then I think I am better
> understanding your suggested solutions. Thank you again for your thoughtful
> replies.
>  
> - Justin
>  
> #############################################################
>  
> This message is sent to you because you are subscribed to
> the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to:  
> Send administrative queries to  
>  


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Ke Carlton-3
Yes; this is correct — it's only the data of ->find() that is copied and
calling the method that causes a lock (or something like outputting
cachestore->asstring).

It's also important to remember that the more keys a map has the longer
each find is going to take and the thread is also locked during the look
up. So if each look up takes 100 - 300 micros (which seems slow) that's 10
- 30 milliseconds per thread which may unacceptable depending on load.

Also if you have Datasource installed consider using hashtable instead of
map — it's faster and appears to handle concurrent writes better; see below
(all times in micros):

Sequential finds:

0 items: hashtable 27 vs map 20
200 items: hashtable 26 vs map 28
324 items: hashtable 24 vs map 28
443 items: hashtable 25 vs map 30
562 items: hashtable 22 vs map 28
809 items: hashtable 23 vs map 28
1417 items: hashtable 23 vs map 30
2220 items: hashtable 24 vs map 31
6660 items: hashtable 23 vs map 32
8880 items: hashtable 24 vs map 34
11100 items: hashtable 25 vs map 33
31080 items: hashtable 31 vs map 37
188963 items: hashtable 57 vs map 78
193403 items: hashtable 43 vs map 68
197843 items: hashtable 36 vs map 64

Concurrent finds + writes + removals:

188 items: hashtable 149 vs map 225
976 items: hashtable 198 vs map 246
1094 items: hashtable 203 vs map 244
2219 items: hashtable 194 vs map 212
0 items: hashtable 1308 vs map 2275
189 items: hashtable 1403 vs map 2464
1552 items: hashtable 2013 vs map 2565
1552 items: hashtable 2013 vs map 2565
2231 items: hashtable 1813 vs map 2440
0 items: hashtable 214 vs map 730
188 items: hashtable 249 vs map 660
300 items: hashtable 307 vs map 574
985 items: hashtable 451 vs map 379
3582 items: hashtable 385 vs map 510
4602 items: hashtable 1333 vs map 2344
6739 items: hashtable 1781 vs map 2520

Ke

On 16 January 2015 at 06:11, Brad Lindsay <[hidden email]> wrote:

> I’m pretty sure Jolle’s assertion that “Each time you do stageStoreCache
> -> find(’...') you copy 5+ MB of data to the current thread” is wrong.
> Thread objects don’t produce a full copy of themselves each time you try to
> use them - they only copy the data to the thread object that you send and
> copy the data from the thread object that it sends back.
>
> Calling [stageStoreCache->find('key')] copies the string object “key” to
> the thread object and then copies the return value from the “find” out to
> the current thread.
>
> That being said, if those top-level objects stored are large, then that’s
> a large amount of data being copied. If what you need can be further
> drilled down to, well, see mine and Ke’s earlier suggestions.
>
> Brad
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Jolle Carlestam-2
In reply to this post by Brad Lindsay
15 jan 2015 kl. 18:11 skrev Brad Lindsay <[hidden email]>:

> I’m pretty sure Jolle’s assertion that “Each time you do stageStoreCache -> find(’...') you copy 5+ MB of data to the current thread” is wrong.

I am pretty sure I am wrong too.
Sorry for adding confusion and misdirections to this thread. I will make sure I have better research next time I utter something on this list.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Bil Corry-3
My solution to being wrong:  patch the code so that I'm right.


- Bil

On Fri, Jan 16, 2015 at 11:50 AM, Jolle Carlestam <[hidden email]>
wrote:

> 15 jan 2015 kl. 18:11 skrev Brad Lindsay <[hidden email]>:
>
> > I’m pretty sure Jolle’s assertion that “Each time you do stageStoreCache
> -> find(’...') you copy 5+ MB of data to the current thread” is wrong.
>
> I am pretty sure I am wrong too.
> Sorry for adding confusion and misdirections to this thread. I will make
> sure I have better research next time I utter something on this list.
>
> HDB
> Jolle
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>   the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Jolle Carlestam-2
16 jan 2015 kl. 15:42 skrev Bil Corry <[hidden email]>:

> My solution to being wrong:  patch the code so that I'm right.
>
>
> - Bil

And for those on the list that’s not aware of this already. Bil works for Paypal…
A company known for never being wrong.




(They do have a lot of patched code though.)

 :-)
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Concurrent Read Access to Thread Object that Emulates Map

Justin Dennis-3
In reply to this post by Justin Dennis-3
Hi All -

Brad has been helping us on the side with tackling this issue. It now
appears that the slow down was caused a different thread object that
functions in a very similar way (map, copies, all the bad stuff we've
talked about). It must've hit a critical mass recently where it started
bogging down. We are running a very small site by web server standards –
internal users only with a maximum of 15-20 or so concurrent users. Each of
these users has a page open with a 3-second javascript XHR poll on this
thread object.

Still working through this, but I thought I owed you all an update after
your helpful feedback. Thank you.

- Justin

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>