Performance Hit Using Custom Types?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Performance Hit Using Custom Types?

Michael Jones-15
Hello,

I have been using custom types to provide object references to
different elements of our system, and for the most part this has been
working very well. However, I am noticing on one of our reports that
I'm taking a serious performance hit for a relatively small amount of
data. Here's what's happening:

I have an inline that selects a bunch of transactions from the
database. The query is only returning < 5,000 rows. Then in my inline,
I loop through the records and create a reference to the object like
this:

local('Transaction' = SV_AccountTransaction);

I assign a few properties based on the data returned and then insert
the object into an array with #TxList->Insert(#Transaction); so that I
can loop through the array later and reference each transaction easily
using #Transaction->Amount, #Transaction->Description, etc.  The
problem is, the page takes about 7-8 times longer to load than if I
just insert the results into the array directly.

Any ideas? Am I going about the whole object-oriented idea wrong?

Michael Jones
Senior Developer
SkyVantage Airline Solutions

Phone: (801) 938-5396
Email: [hidden email]
Website: http://www.skyvantage.com/

--

This message may contain confidential and/or privileged information of
SkyVantage Corporation and  its affiliated companies.  If you are not
the intended recipient, please (i) do not disclose,  copy, distribute,
or use this information, (ii) advise the sender by return e-mail, and
(iii) delete  all copies from your computer.  Your cooperation is
greatly appreciated.

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Performance Hit Using Custom Types?

Ke Carlton-3
On 26 February 2010 21:45, Michael Jones <[hidden email]> wrote:
>
> I assign a few properties based on the data returned and then insert
> the object into an array with #TxList->Insert(#Transaction); so that I
> can loop through the array later and reference each transaction easily
> using #Transaction->Amount, #Transaction->Description, etc.  The
> problem is, the page takes about 7-8 times longer to load than if I
> just insert the results into the array directly.
>
> Any ideas? Am I going about the whole object-oriented idea wrong?

I'd check that the type's defined as a -prototype and perhaps look at
assigning the data within the onCreate definition.

It's worth noting that when performing mass processing your fastest
route is always likely to be the default datatypes.

Ke

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Performance Hit Using Custom Types?

Michael Jones-15
Thanks for the response.  I actually am assigning the params in the
onCreate by passing them when I create the object, but I had actually
temporarily removed that and tried doing it the other way to see if it
made a difference.

I'll look into the -prototype idea, and if all else fails I'll
probably just use a map -- it's not imperative that I use these types
for this report, just was trying to standardize how we access the
data.


On Fri, Feb 26, 2010 at 3:04 PM, Ke Carlton <[hidden email]> wrote:

> On 26 February 2010 21:45, Michael Jones <[hidden email]> wrote:
>>
>> I assign a few properties based on the data returned and then insert
>> the object into an array with #TxList->Insert(#Transaction); so that I
>> can loop through the array later and reference each transaction easily
>> using #Transaction->Amount, #Transaction->Description, etc.  The
>> problem is, the page takes about 7-8 times longer to load than if I
>> just insert the results into the array directly.
>>
>> Any ideas? Am I going about the whole object-oriented idea wrong?
>
> I'd check that the type's defined as a -prototype and perhaps look at
> assigning the data within the onCreate definition.
>
> It's worth noting that when performing mass processing your fastest
> route is always likely to be the default datatypes.
>
> Ke
>
> --
> This list is a free service of LassoSoft: http://www.LassoSoft.com/
> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
> Manage your subscription: http://www.ListSearch.com/Lasso/
>
>
>

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Performance Hit Using Custom Types?

Tim Taplin
In reply to this post by Michael Jones-15
two levels to look at.

first in your custom type, pass the properties in to the onCreate right off the bat so you dont have to round trip it so many times.
        local('Transaction' = SV_AccountTransaction(-amount=field('amount'), -description=field('description')) );

second and I'd have to test this to see how much it matters, but I would probably have a custom tag that would take the query results as an array or map parameter and return the completed array of transaction objects. There are significant performance enhancements to having the looping done in a custom tag.

All that being said, there's a cost to taking elements out of one array, creating a new object and inserting it into a new array. Unless you are accessing the same item hundreds of times before letting it go or performing some additional operations to create the properties such as on the fly calculations, string concat (full name from first/, last), or subset creations, its probably not worth the overhead to use #transaction->amount instead of #transaction->find('amount') or #transaction->get(2) if you dont want to use a map.

Tim Taplin
 
On Feb 26, 2010, at 2:45 PM, Michael Jones wrote:

> Hello,
>
> I have been using custom types to provide object references to
> different elements of our system, and for the most part this has been
> working very well. However, I am noticing on one of our reports that
> I'm taking a serious performance hit for a relatively small amount of
> data. Here's what's happening:
>
> I have an inline that selects a bunch of transactions from the
> database. The query is only returning < 5,000 rows. Then in my inline,
> I loop through the records and create a reference to the object like
> this:
>
> local('Transaction' = SV_AccountTransaction);
>
> I assign a few properties based on the data returned and then insert
> the object into an array with #TxList->Insert(#Transaction); so that I
> can loop through the array later and reference each transaction easily
> using #Transaction->Amount, #Transaction->Description, etc.  The
> problem is, the page takes about 7-8 times longer to load than if I
> just insert the results into the array directly.
>
> Any ideas? Am I going about the whole object-oriented idea wrong?
>
> Michael Jones
> Senior Developer
> SkyVantage Airline Solutions
>
> Phone: (801) 938-5396
> Email: [hidden email]
> Website: http://www.skyvantage.com/
>
> --
>
> This message may contain confidential and/or privileged information of
> SkyVantage Corporation and  its affiliated companies.  If you are not
> the intended recipient, please (i) do not disclose,  copy, distribute,
> or use this information, (ii) advise the sender by return e-mail, and
> (iii) delete  all copies from your computer.  Your cooperation is
> greatly appreciated.
>
> --
> This list is a free service of LassoSoft: http://www.LassoSoft.com/
> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
> Manage your subscription: http://www.ListSearch.com/Lasso/
>
>


--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Performance Hit Using Custom Types?

Michael Jones-15
Hey Tim,

Thanks for the feedback. I'll probably just stick with a map for
simplicity.. like I mentioned on one of the other replies I did
initially have it passing the properties in as part of the creation,
but tried both ways to see if it helped any. I actually have a custom
tag inside the object that takes the map of records as a parameter and
sets up the object based on that.

I'm not sure I follow how the performance would be vastly improved in
passing the results through a map to a custom tag vs. skipping the
extra step and just doing the loop directly in the inline. I could see
how it might help to preload the tag into memory to gain a small
improvement, but I'm guessing the hit would still occur when actually
creating the custom type object.

I figure for this it's mostly just a question of convention rather
than necessity, but good to know that I'm not too far off in what I'm
trying to do.


Michael Jones
Senior Developer
SkyVantage Airline Solutions

Phone: (801) 938-5396
Email: [hidden email]
Website: http://www.skyvantage.com/

--

This message may contain confidential and/or privileged information of
SkyVantage Corporation and  its affiliated companies.  If you are not
the intended recipient, please (i) do not disclose,  copy, distribute,
or use this information, (ii) advise the sender by return e-mail, and
(iii) delete  all copies from your computer.  Your cooperation is
greatly appreciated.



On Fri, Feb 26, 2010 at 3:22 PM, Tim Taplin <[hidden email]> wrote:

> two levels to look at.
>
> first in your custom type, pass the properties in to the onCreate right off the bat so you dont have to round trip it so many times.
>        local('Transaction' = SV_AccountTransaction(-amount=field('amount'), -description=field('description')) );
>
> second and I'd have to test this to see how much it matters, but I would probably have a custom tag that would take the query results as an array or map parameter and return the completed array of transaction objects. There are significant performance enhancements to having the looping done in a custom tag.
>
> All that being said, there's a cost to taking elements out of one array, creating a new object and inserting it into a new array. Unless you are accessing the same item hundreds of times before letting it go or performing some additional operations to create the properties such as on the fly calculations, string concat (full name from first/, last), or subset creations, its probably not worth the overhead to use #transaction->amount instead of #transaction->find('amount') or #transaction->get(2) if you dont want to use a map.
>
> Tim Taplin
>
> On Feb 26, 2010, at 2:45 PM, Michael Jones wrote:
>
>> Hello,
>>
>> I have been using custom types to provide object references to
>> different elements of our system, and for the most part this has been
>> working very well. However, I am noticing on one of our reports that
>> I'm taking a serious performance hit for a relatively small amount of
>> data. Here's what's happening:
>>
>> I have an inline that selects a bunch of transactions from the
>> database. The query is only returning < 5,000 rows. Then in my inline,
>> I loop through the records and create a reference to the object like
>> this:
>>
>> local('Transaction' = SV_AccountTransaction);
>>
>> I assign a few properties based on the data returned and then insert
>> the object into an array with #TxList->Insert(#Transaction); so that I
>> can loop through the array later and reference each transaction easily
>> using #Transaction->Amount, #Transaction->Description, etc.  The
>> problem is, the page takes about 7-8 times longer to load than if I
>> just insert the results into the array directly.
>>
>> Any ideas? Am I going about the whole object-oriented idea wrong?
>>
>> Michael Jones
>> Senior Developer
>> SkyVantage Airline Solutions
>>
>> Phone: (801) 938-5396
>> Email: [hidden email]
>> Website: http://www.skyvantage.com/
>>
>> --
>>
>> This message may contain confidential and/or privileged information of
>> SkyVantage Corporation and  its affiliated companies.  If you are not
>> the intended recipient, please (i) do not disclose,  copy, distribute,
>> or use this information, (ii) advise the sender by return e-mail, and
>> (iii) delete  all copies from your computer.  Your cooperation is
>> greatly appreciated.
>>
>> --
>> This list is a free service of LassoSoft: http://www.LassoSoft.com/
>> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
>> Manage your subscription: http://www.ListSearch.com/Lasso/
>>
>>
>
>
> --
> This list is a free service of LassoSoft: http://www.LassoSoft.com/
> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
> Manage your subscription: http://www.ListSearch.com/Lasso/
>
>
>

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Performance Hit Using Custom Types?

Tim Taplin
heheh, yeah, I got busy typing that up and better answers made their way in line before I finished.

The key to the use of ctag would be to avoid the records loop and just pass the records_map or records_array into the ctag.
Then the preloaded tag and type would be faster than a page based records loop and inline processing.

Like I said, it could be worth testing if you can precalculate stuff you'll need later, really useful if you have to do lots of little calcs later.

Tim

On Feb 26, 2010, at 3:53 PM, Michael Jones wrote:

> Hey Tim,
>
> Thanks for the feedback. I'll probably just stick with a map for
> simplicity.. like I mentioned on one of the other replies I did
> initially have it passing the properties in as part of the creation,
> but tried both ways to see if it helped any. I actually have a custom
> tag inside the object that takes the map of records as a parameter and
> sets up the object based on that.
>
> I'm not sure I follow how the performance would be vastly improved in
> passing the results through a map to a custom tag vs. skipping the
> extra step and just doing the loop directly in the inline. I could see
> how it might help to preload the tag into memory to gain a small
> improvement, but I'm guessing the hit would still occur when actually
> creating the custom type object.
>
> I figure for this it's mostly just a question of convention rather
> than necessity, but good to know that I'm not too far off in what I'm
> trying to do.
>
>
> Michael Jones
> Senior Developer
> SkyVantage Airline Solutions
>
> Phone: (801) 938-5396
> Email: [hidden email]
> Website: http://www.skyvantage.com/
>
> --
>
> This message may contain confidential and/or privileged information of
> SkyVantage Corporation and  its affiliated companies.  If you are not
> the intended recipient, please (i) do not disclose,  copy, distribute,
> or use this information, (ii) advise the sender by return e-mail, and
> (iii) delete  all copies from your computer.  Your cooperation is
> greatly appreciated.
>
>
>
> On Fri, Feb 26, 2010 at 3:22 PM, Tim Taplin <[hidden email]> wrote:
>> two levels to look at.
>>
>> first in your custom type, pass the properties in to the onCreate right off the bat so you dont have to round trip it so many times.
>>        local('Transaction' = SV_AccountTransaction(-amount=field('amount'), -description=field('description')) );
>>
>> second and I'd have to test this to see how much it matters, but I would probably have a custom tag that would take the query results as an array or map parameter and return the completed array of transaction objects. There are significant performance enhancements to having the looping done in a custom tag.
>>
>> All that being said, there's a cost to taking elements out of one array, creating a new object and inserting it into a new array. Unless you are accessing the same item hundreds of times before letting it go or performing some additional operations to create the properties such as on the fly calculations, string concat (full name from first/, last), or subset creations, its probably not worth the overhead to use #transaction->amount instead of #transaction->find('amount') or #transaction->get(2) if you dont want to use a map.
>>
>> Tim Taplin
>>
>> On Feb 26, 2010, at 2:45 PM, Michael Jones wrote:
>>
>>> Hello,
>>>
>>> I have been using custom types to provide object references to
>>> different elements of our system, and for the most part this has been
>>> working very well. However, I am noticing on one of our reports that
>>> I'm taking a serious performance hit for a relatively small amount of
>>> data. Here's what's happening:
>>>
>>> I have an inline that selects a bunch of transactions from the
>>> database. The query is only returning < 5,000 rows. Then in my inline,
>>> I loop through the records and create a reference to the object like
>>> this:
>>>
>>> local('Transaction' = SV_AccountTransaction);
>>>
>>> I assign a few properties based on the data returned and then insert
>>> the object into an array with #TxList->Insert(#Transaction); so that I
>>> can loop through the array later and reference each transaction easily
>>> using #Transaction->Amount, #Transaction->Description, etc.  The
>>> problem is, the page takes about 7-8 times longer to load than if I
>>> just insert the results into the array directly.
>>>
>>> Any ideas? Am I going about the whole object-oriented idea wrong?
>>>
>>> Michael Jones
>>> Senior Developer
>>> SkyVantage Airline Solutions
>>>
>>> Phone: (801) 938-5396
>>> Email: [hidden email]
>>> Website: http://www.skyvantage.com/
>>>
>>> --
>>>
>>> This message may contain confidential and/or privileged information of
>>> SkyVantage Corporation and  its affiliated companies.  If you are not
>>> the intended recipient, please (i) do not disclose,  copy, distribute,
>>> or use this information, (ii) advise the sender by return e-mail, and
>>> (iii) delete  all copies from your computer.  Your cooperation is
>>> greatly appreciated.
>>>
>>> --
>>> This list is a free service of LassoSoft: http://www.LassoSoft.com/
>>> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
>>> Manage your subscription: http://www.ListSearch.com/Lasso/
>>>
>>>
>>
>>
>> --
>> This list is a free service of LassoSoft: http://www.LassoSoft.com/
>> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
>> Manage your subscription: http://www.ListSearch.com/Lasso/
>>
>>
>>
>
> --
> This list is a free service of LassoSoft: http://www.LassoSoft.com/
> Search the list archives: http://www.ListSearch.com/Lasso/Browse/
> Manage your subscription: http://www.ListSearch.com/Lasso/
>
>


--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/


Reply | Threaded
Open this post in threaded view
|

Re: Performance Hit Using Custom Types?

James Harvard
In reply to this post by Michael Jones-15
>I have an inline that selects a bunch of transactions from the
>database. The query is only returning < 5,000 rows. Then in my inline,
>I loop through the records and create a reference to the object like
>this:
>
>local('Transaction' = SV_AccountTransaction);

Creating the new type is almost certainly the bottleneck. As Ke suggested you could try employing the prototype feature or you could do something like this to reuse your custom type instead of create a new instance each time:

local( 't' = SV_AccountTransaction, 'TxList' = array );
rows;
        #t->load;
        #TxList->insert( #t );
/rows;

(Hope I've understood correctly what you're doing!)

The principle at work here is that copying an instance of a ctype is faster than creating a new instance, but that is also why using prototypes should be faster so I don't know whether you would find a performance difference between the two routes.

James

--
This list is a free service of LassoSoft: http://www.LassoSoft.com/
Search the list archives: http://www.ListSearch.com/Lasso/Browse/
Manage your subscription: http://www.ListSearch.com/Lasso/