Advice on automatic removal of files

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Advice on automatic removal of files

Jolle Carlestam-3
Lasso 9, both Centos and OSX.

I'm in need of an automatic routine that would remove files from a given directory after they're for example 30 minutes old.

Suppose ways to do it would be to trigger the removal with a delay when the file is created or to have some kind of recurring event that checks all files in the directory and deletes the ones that are older than 30 minutes.

Any advice on best practice is appreciated.

HDB
Jolle
#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Mark Palmer
You could do it on the command line with something like this:

find . -mmin -30 -type f -execdir mv {} /blah/blah/_archived_files/ \;

This is moving rather than deleting just to be safe. The "." is the current directory so will need replacing with your full path.

I guess Lasso 9 could call this periodically or a cron job could.



On 14 Jul 2011, at 10:06, Jolle Carlestam wrote:

> Lasso 9, both Centos and OSX.
>
> I'm in need of an automatic routine that would remove files from a given directory after they're for example 30 minutes old.
>
> Suppose ways to do it would be to trigger the removal with a delay when the file is created or to have some kind of recurring event that checks all files in the directory and deletes the ones that are older than 30 minutes.
>
> Any advice on best practice is appreciated.
>
> HDB
> Jolle



Regards

Mark Palmer
E: [hidden email]
T: 01902 620500
W: www.pageworks.co.uk




#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

stevepiercy
In reply to this post by Jolle Carlestam-3
I've programmatically created a Lasso event to fire off at a
designated time in the future with event_schedule in Lasso 8.x,
but I don't see it in 9.

http://www.lassosoft.com/lassoDocs/languageReference/obj/Event_Schedule?v=8
http://www.lassosoft.com/lassoDocs/languageReference/obj/Event_Schedule?v=9

That could be a documentation oversight, but I don't see
anything in the L9 Admin either, so it is probably gone.

I think the preferred method in 9 is either to manage it through
os_process and execute a one-time cron job (challenging,
involves permissions) or set up a cron job to run periodically
through the command line (easy, set it and forget it).

There's many ways to skin that cat.  Since my *nix skillz are
not mad, I would probably do something where I alternate where
files are written based on the server's minutes time.

h:00 - h:29 -> write file to /files/0_29
h:30 - h:59 -> write file to /files/30_59

Then have two cron jobs that run alternately every 30 minutes to
delete all files within each directory.

59  *   *   *   *   /usr/bin/delete_from_0_29
29  *   *   *   *   /usr/bin/delete_from_30_59

Then you write a bash script that executes the delete command.

--steve


On 7/14/11 at 9:06 AM, [hidden email] (Jolle
Carlestam) pronounced:

>Lasso 9, both Centos and OSX.
>
>I'm in need of an automatic routine that would remove files
>from a given directory after they're for example 30 minutes old.
>
>Suppose ways to do it would be to trigger the removal with a
>delay when the file is created or to have some kind of
>recurring event that checks all files in the directory and
>deletes the ones that are older than 30 minutes.
>
>Any advice on best practice is appreciated.
>
>HDB
>Jolle
>#############################################################
>This message is sent to you because you are subscribed to
>the mailing list <[hidden email]>.
>To unsubscribe, E-mail to: <[hidden email]>
>To switch to the DIGEST mode, E-mail to <[hidden email]>
>To switch to the INDEX mode, E-mail to <[hidden email]>
>Send administrative queries to  <[hidden email]>
>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
-- --
Steve Piercy               Web Site Builder              
Soquel, CA
<[hidden email]>                  <http://www.StevePiercy.com/>


#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Jolle Carlestam-3
In reply to this post by Jolle Carlestam-3
14 jul 2011 kl. 11.06 skrev Jolle Carlestam:

> Lasso 9, both Centos and OSX.
>
> I'm in need of an automatic routine that would remove files from a given directory after they're for example 30 minutes old.
>
> Suppose ways to do it would be to trigger the removal with a delay when the file is created or to have some kind of recurring event that checks all files in the directory and deletes the ones that are older than 30 minutes.
>
> Any advice on best practice is appreciated.

Thanks Steve and Mark for your input but I think you over complicate it.
Should be no need to do os_process, command line or cron jobs. The actual code using Lasso 9 is simple:

var(file_temp = null)
var(compare_date = date)
$compare_date -> subtract(-minute = 30)
iterate(file_listdirectory('/my_tmp/'))
        $file_temp = file('/my_tmp/' + loop_value)
        if($file_temp -> modificationDate < $compare_date)
                $file_temp -> delete
                'The file ' + loop_value + ' was deleted'
        else
                'The file ' + loop_value + ' was not touched, for now'
        /if
        '<br>'
/iterate

What's left is to decide how to run it. And as Steve pointed out there is no longer an event scheduler in Lasso 9. The replacement is not to do cron jobs, it's to use threads and active_tick:
http://www.lassosoft.com/Language-Guide-Threading#heading10

I knew about this just wanted input from coders who might have threaded (hihi) this path before.

HDB
Jolle
#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

stevepiercy
On 7/14/11 at 10:08 AM, [hidden email] (Jolle
Carlestam) pronounced:

>14 jul 2011 kl. 11.06 skrev Jolle Carlestam:
>
>>Lasso 9, both Centos and OSX.
>>
>>I'm in need of an automatic routine that would remove files from a given directory
>after they're for example 30 minutes old.
>>
>>Suppose ways to do it would be to trigger the removal with a delay when the file
>is created or to have some kind of recurring event that checks
>all files in the directory and deletes the ones that are older
>than 30 minutes.
>>
>>Any advice on best practice is appreciated.
>
>Thanks Steve and Mark for your input but I think you over complicate it.
>Should be no need to do os_process, command line or cron jobs.
>The actual code using Lasso 9 is simple:
>
>var(file_temp = null)
>var(compare_date = date)
>$compare_date -> subtract(-minute = 30)
>iterate(file_listdirectory('/my_tmp/'))
>$file_temp = file('/my_tmp/' + loop_value)
>if($file_temp -> modificationDate < $compare_date)
>$file_temp -> delete
>'The file ' + loop_value + ' was deleted'
>else
>'The file ' + loop_value + ' was not touched, for now'
>/if
>'<br>'
>/iterate
>
>What's left is to decide how to run it. And as Steve pointed
>out there is no longer an event scheduler in Lasso 9. The
>replacement is not to do cron jobs, it's to use threads and active_tick:
>http://www.lassosoft.com/Language-Guide-Threading#heading10

Sure, you could have cron call that Lasso 9 script, too.  The
cat has less skin now.

Personally I prefer cron since it would not generate a new
thread.  Maybe that is splitting hairs and not much of an
argument.  Also I don't know what would happen to the thread if
you get an ISE with L9, but you can try it and find out.

--steve

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
-- --
Steve Piercy               Web Site Builder              
Soquel, CA
<[hidden email]>                  <http://www.StevePiercy.com/>


#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Brad Lindsay-2
In reply to this post by Jolle Carlestam-3
Depending on your setup and environment, doing either is fine. Personally, I would probably be tempted to do both.

When the file is created, I'd create and asynchronous thread whose first thing it did was [sleep] the specified time before running the code to delete the file. I would then probably also have a daily script that I would use cron to get to run just to be sure that I had cleaned up after myself by deleting all the files that are older than an hour.

Brad


On Jul 14, 2011, at 5:06 AM, Jolle Carlestam wrote:

> Lasso 9, both Centos and OSX.
>
> I'm in need of an automatic routine that would remove files from a given directory after they're for example 30 minutes old.
>
> Suppose ways to do it would be to trigger the removal with a delay when the file is created or to have some kind of recurring event that checks all files in the directory and deletes the ones that are older than 30 minutes.
>
> Any advice on best practice is appreciated.
>
> HDB
> Jolle
> #############################################################
> This message is sent to you because you are subscribed to
>  the mailing list <[hidden email]>.
> To unsubscribe, E-mail to: <[hidden email]>
> To switch to the DIGEST mode, E-mail to <[hidden email]>
> To switch to the INDEX mode, E-mail to <[hidden email]>
> Send administrative queries to  <[hidden email]>
>


#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Jolle Carlestam-3
In reply to this post by Jolle Carlestam-3
14 jul 2011 kl. 12.08 skrev Jolle Carlestam:

> What's left is to decide how to run it. And as Steve pointed out there is no longer an event scheduler in Lasso 9. The replacement is not to do cron jobs, it's to use threads and active_tick:
> http://www.lassosoft.com/Language-Guide-Threading#heading10
>
> I knew about this just wanted input from coders who might have threaded (hihi) this path before.


Knowing about something is not the same as knowing how to use it.
I have anyway made the following attempt:
define delete_tmp => thread {

        data
                public paths::map = map,
                public initiated::date,
                public interval::integer = 1800

        public onCreate() => {
                .'initiated' = date
        }

        public setpath(path::string, expiresecs::integer) => {
                .'paths' -> insert(#path = #expiresecs)
        }

        public removepath(path::string) => {
                if(.'paths' -> find(#path)) => {
                        .'paths' -> remove(#path)
                }
        }

        public setinterval(interval::integer) => {
                .'interval' = #interval
        }

        public paths() => .'paths'

        public getinterval() => .'interval'

        public active_tick() => {
                if(.'paths' -> size) => {
                        local(file_temp = null)
                        local(compare_date = null)
                        local(all_gone = true)
                        iterate(.'paths', local(p)) => {
                                #compare_date = date
                                #compare_date -> subtract(-minute = #p -> value)
                                iterate(file_listdirectory(#p -> name)) => {
                                        #file_temp = file(#p -> name + loop_value)
                                        if(#file_temp -> modificationDate < #compare_date) => {
                                                #file_temp -> delete
                                                stdoutnl('The file ' + #p -> name + loop_value + ' was deleted')
                                        else
                                                stdoutnl('The file ' + #p -> name + loop_value + ' was not touched, for now')
                                                #all_gone = false
                                        }
                                }
                        }

                        #all_gone ? .'interval' = 1800

                }

                return .'interval'
        }
}

The idea being that this could be used to delete files from a number of different temp directories.
Since it's a thread it seems that it knows nothing about web root so absolute paths from server root need to be sent to it.

In the file that creates the temporary files I have this code:
// set path for the future deletion of the file
if(delete_tmp -> paths -> find(file_forceRoot + 'my_tmp/') -> size == 0) => {
        delete_tmp -> setpath(file_forceRoot + 'my_tmp/', 30)
        delete_tmp -> setinterval(120)
}

The default interval if left unused is 1800 seconds meaning that the thread will tick every half hour. In the above call I increase that to every two minutes. It will continue to check every two minutes until no more files for deletion can be found. By that point it will go back to the half hour tick.
The stdoutnl rows should of course be commented out for daily use.

Still interested in potential flaws or optimization possibilities.


Some questions.
Where do I put my thread definition? Need it go into LassoStartup or can it reside in LassoLibraries?

Is there someway of knowing if a thread object is created already?

HDB
Jolle
#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

decorior
In reply to this post by stevepiercy
The event_schedule is gone?

This will be a major issue for us. We need to setup an expiration as you describe but it also passes parameters. Plus it seems we will now have to write all of the associated logging, etc.

I hope it is still there, otherwise this is the kind of migration that will add another two weeks to the development migration.


On Jul 14, 2011, at 3:48 AM, Steve Piercy - Web Site Builder wrote:

> I've programmatically created a Lasso event to fire off at a designated time in the future with event_schedule in Lasso 8.x, but I don't see it in 9.
>
> http://www.lassosoft.com/lassoDocs/languageReference/obj/Event_Schedule?v=8
> http://www.lassosoft.com/lassoDocs/languageReference/obj/Event_Schedule?v=9
>
> That could be a documentation oversight, but I don't see anything in the L9 Admin either, so it is probably gone.
>
> I think the preferred method in 9 is either to manage it through os_process and execute a one-time cron job (challenging, involves permissions) or set up a cron job to run periodically through the command line (easy, set it and forget it).
>
> There's many ways to skin that cat.  Since my *nix skillz are not mad, I would probably do something where I alternate where files are written based on the server's minutes time.
>
> h:00 - h:29 -> write file to /files/0_29
> h:30 - h:59 -> write file to /files/30_59
>
> Then have two cron jobs that run alternately every 30 minutes to delete all files within each directory.
>
> 59  *   *   *   *   /usr/bin/delete_from_0_29
> 29  *   *   *   *   /usr/bin/delete_from_30_59
>
> Then you write a bash script that executes the delete command.
>
> --steve
>
>
> On 7/14/11 at 9:06 AM, [hidden email] (Jolle Carlestam) pronounced:
>
>> Lasso 9, both Centos and OSX.
>>
>> I'm in need of an automatic routine that would remove files from a given directory after they're for example 30 minutes old.
>>
>> Suppose ways to do it would be to trigger the removal with a delay when the file is created or to have some kind of recurring event that checks all files in the directory and deletes the ones that are older than 30 minutes.
>>
>> Any advice on best practice is appreciated.
>>
>> HDB
>> Jolle
>> #############################################################
>> This message is sent to you because you are subscribed to
>> the mailing list <[hidden email]>.
>> To unsubscribe, E-mail to: <[hidden email]>
>> To switch to the DIGEST mode, E-mail to <[hidden email]>
>> To switch to the INDEX mode, E-mail to <[hidden email]>
>> Send administrative queries to  <[hidden email]>
>>
>
> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> Steve Piercy               Web Site Builder               Soquel, CA
> <[hidden email]>                  <http://www.StevePiercy.com/>
>
>
> #############################################################
> This message is sent to you because you are subscribed to
> the mailing list <[hidden email]>.
> To unsubscribe, E-mail to: <[hidden email]>
> To switch to the DIGEST mode, E-mail to <[hidden email]>
> To switch to the INDEX mode, E-mail to <[hidden email]>
> Send administrative queries to  <[hidden email]>
>

Deco Rior
[hidden email]

O:(720) 207-2806
F:(303) 942-7417

TennisSource.Net
software to power your tennis facility


This electronic mail message is intended exclusively for the individual or entity to which it is addressed. This message, together with any attachment, may contain confidential and privileged information.   Any unauthorized review, use, printing, copying, retention, disclosure or distribution is strictly prohibited. If you have received this message in error, please immediately advise the sender by reply email message to the sender and delete all copies of this message. Thank you


#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Jolle Carlestam-3
14 jul 2011 kl. 15.20 skrev Deco Rior:

> The event_schedule is gone?

Yes, but I would say that it's replaced with a better option.
The way to go is to use thread objects instead.

HDB
Jolle

#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

stevepiercy
On 7/14/11 at 1:26 PM, [hidden email] (Jolle
Carlestam) pronounced:

>14 jul 2011 kl. 15.20 skrev Deco Rior:
>
>>The event_schedule is gone?
>
>Yes, but I would say that it's replaced with a better option.
>The way to go is to use thread objects instead.

Using the better option of thread objects, how would you handle
the situation where it is a one-time event that must be
scheduled relative to some user interaction, and after the
thread object is created, the server restarts or has an ISE?

--steve

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
-- --
Steve Piercy               Web Site Builder              
Soquel, CA
<[hidden email]>                  <http://www.StevePiercy.com/>


#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Jolle Carlestam-3
14 jul 2011 kl. 15.32 skrev Steve Piercy - Web Site Builder:

>> 14 jul 2011 kl. 15.20 skrev Deco Rior:
>>
>>> The event_schedule is gone?
>>
>> Yes, but I would say that it's replaced with a better option.
>> The way to go is to use thread objects instead.
>
> Using the better option of thread objects, how would you handle the situation where it is a one-time event that must be scheduled relative to some user interaction, and after the thread object is created, the server restarts or has an ISE?
>
> --steve

(Sorry for not replying faster.)
By creating my oven event scheduler.
For Lassoers that don't want to do that here's a suggested event_schedule thread.
It uses a table to store events and will poll the event_schedule table at a given interval. Default is once a minute but the value can be changed by a setting.
Events are created by the event_schedule -> create method. Example:
event_schedule -> create('http://example.com',
                -start = date,
                -end = null,
                -delay = 4,
                -restart = true,
                -username = string,
                -password = string,
                -silent = false // Default is true. If true will return no results from a call
)

I have refrained from providing a ready made form as I see that more for each developers task.

if username and or password is provided they will be stored encrypted in the table. Simple blowfish but still keep prying eyes from seeing cleartext values.

To check stored events use the method event_schedule -> events. It will return a map of maps with all records in the event_schedule table.

Removing methods before their time is done using event_schedule -> remove('id_value')

There're also some other methods provided. See the code for info.

I scanned the Lasso 8.5 reference for inspiration. http://reference.lassosoft.com/Reference.LassoApp?[Event_Schedule]

To start using it create an event_schedule table in your preferred database. SQL code is supplied in the thread comments
Place a file with the event_schedule code in LassoStartup and restart Lasso server.

The provided thread should be considered experimental so far. It has not in any way been battle tested. I welcome any suggestions of improvement or bug calls.
HDB
Jolle

-----------------------
Code:
<?LassoScript

/**!
event_schedule
Experimental Event Schedule thread for Lasso 9
Will at a set interval check the event_schedule table for urls to run. If found then use include_url to call it.
If left unprovoked will do repeat check every minute (60 seconds).

### Methods:
Set the database where the event_schedule table resides
event_schedule -> setdatabase('test')

Create an event that is stored in the event_schedule table and run at the given interval. If no delay value is provided it will run once. Due to a present bug in Lasso 9 thread handling all params must be provided, even the optional ones.
event_schedule -> create('http://example.com',
                -start = date,
                -end = null,
                -delay = 4,
                -restart = true,
                -username = string,
                -password = string,
                -silent = false // Default is true. If true will return no results from a call
)

Will return a map with all events in the event_schedule table
event_schedule -> events

Changes the interval used to poll the event_schedule table. Default is 60 seconds
event_schedule -> setinterval(120) // will change the check param to every two minutes.

Retrieve the interval used for polling the event_schedule table
event_schedule -> getinterval

Removes the event with the given id from the event_schedule table
event_schedule -> remove('id_value')

### SQL for creating the event_schedule table
CREATE TABLE `event_schedule` (
  `event_url` varchar(255) CHARACTER SET utf8 COLLATE utf8_swedish_ci DEFAULT NULL COMMENT 'The URL of the page which should be executed at the time set by the other keyword/value parameters.',
  `event_start` datetime DEFAULT NULL COMMENT 'The date and time at which to execute the event. By default set to the current date and time.',
  `event_end` datetime DEFAULT NULL COMMENT 'The date and time to stop executing a repeating event.',
  `event_delay` int(9) DEFAULT NULL COMMENT 'The number of minutes to delay between repetitions of a repeating event.',
  `event_restart` enum('Y','N') DEFAULT 'N' COMMENT 'If set to true (Y) then the event will be rescheduled after the server restarts. If not set then the event will not be rescheduled. Defaults to False (N).',
  `event_username` varchar(255) DEFAULT NULL COMMENT 'An optional username which will be used to authenticate the execution of the scheduled event. Stored encrypted in the record.',
  `event_password` varchar(255) DEFAULT NULL COMMENT 'An optional password which will be used to authenticate execution of the scheduled event. Stored encrypted in the record.',
  `next_run_datetime` datetime DEFAULT NULL COMMENT 'Set by the system to the next time this event should be run',
  `server_bootup_datetime` datetime DEFAULT NULL COMMENT 'Date and time for when lasso server was started. Set when the event is created in order to know if an event should be triggered or not in relation to flag set in event_restart. ',
  `id` bigint(30) NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (`id`),
  KEY `next_run_datetime` (`next_run_datetime`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8

2011-07-15      JC      First version.

**/
define event_schedule => thread {

        data
                public initiated::date, // when the thread was initiated. Most likely at Lasso server startup
                public database::string, // Database where to find the event_schedule table
                public interval::integer, // Seconds between each poll of the event_schedule table
                private last_cleanup::date, // date and time for when the last cleanup was performed
                private encrypt_seed::string // seed used to encrypt username and password values

        public onCreate() => {
                .'interval' = 60
                .'initiated' = date(date -> format('%q')) // need to set format to get rid of nasty hidden fractions of seconds
//              .'encrypt_seed' = 'AKJHS726KLKJS6823837sgJDIUERHDGJHSGJHJHGJSHHjhsaje84JJJJEWU48' + string(server_ip) + 'ajd2837628' Not working yet, still looking for a replacement to server_name or server_ip
                .'encrypt_seed' = 'AKJHS726KLKJS6823837sgJDIUERHDGJHSGJHJHGJSHHjhsaje84JJJJEWU48ajd2837628'
        }

/**!
create
Create an event that is stored in the event_schedule table and run at the given interval. If no delay value is provided it will run once. Due to a present bug in Lasso 9 thread handling all params must be provided, even the optional ones.
**/
        public create(
                url::string, // The URL of the page which should be executed at the time set by the other keyword/value parameters.
                start::date = date, // The date and time at which to execute the event. By default set to the current date and time.
                end::any = null, // The date and time to stop executing a repeating event.
                delay::integer = 0, // The number of minutes to delay between repetitions of a repeating event.
                restart::boolean = false, // If set to true (Y) then the event will be rescheduled after the server restarts. If not set then the event will not be rescheduled. Defaults to False (N).
                username::string = string, // An optional username which will be used to authenticate the execution of the scheduled event. Stored encrypted in the record.
                password::string = string, // An optional password which will be used to authenticate execution of the scheduled event. Stored encrypted in the record.
                silent::boolean = true // If false report feedback from exekution
        ) => {

                fail_if(.'database' -> size == 0, -1, 'No database has been set')
                fail_if(#url -> size > 255, -1, 'URL is too long (max 255 chars)')
                fail_if(!valid_date(#start, -strict), -1, 'Start date is not valid')
                fail_if(#end != null && !valid_date(#end, -strict), -1, 'End date is not valid')
                fail_if(#username -> size > 127, -1, 'Username is too long (max 127 chars)')
                fail_if(#password -> size > 127, -1, 'Password is too long (max 127 chars)')

                inline(
                        -database = .'database',
                        -table = 'event_schedule',
                        'event_url' = #url,
                        'event_start' = #start,
                        'event_end' = #end,
                        'event_delay' = (#delay < 1 ? null | #delay),
                        'event_restart' = (#restart ? 'Y' | 'N'),
                        'event_username' = (#username -> size == 0 ? null | encrypt_blowfish(-seed = .'encrypt_seed', #username)->encodehex),
                        'event_password' = (#password -> size == 0 ? null | encrypt_blowfish(-seed = .'encrypt_seed',#password)->encodehex),
                        'next_run_datetime' = #start, // Set by the system to the next time this event should be run.
                        'server_bootup_datetime' = .'initiated', // Date and time for when lasso server was started. Set when the event is created in order to know if an event should be triggered or not in relation to flag set in event_restart.
                        'id' = lasso_uniqueid,
                        -add) => {

                        error_code != 0 ? log_critical('Error while creating event in event schedule: ' + error_msg)
                        !#silent ? return error_msg
                }
        }

        public create(
                url::string,
                -start::date = date,
                -end::any = null,
                -delay::integer = 0,
                -restart::boolean = false,
                -username::string = string,
                -password::string = string,
                -silent::boolean = true
        ) => .create(#url, #start, #end, #delay, #restart, #username, #password, #silent)

        public create(
                url::string,
                -start::date = date,
                -end::any = null,
                -delay::integer = 0,
                -restart::boolean = false
        ) => .create(#url, #start, #end, #delay, #restart, string, string, true)

/**!
remove
Removes the event with the given id from the event_schedule table
**/
        public remove(
                id::string
        ) => {
                fail_if(.'database' -> size == 0, -1, 'No database has been set')

                inline(
                        -database = .'database',
                        -table = 'event_schedule',
                        -keyfield = 'id',
                        -keyvalue = #id,
                        -delete) => {
                        error_code != 0 ? log_critical('Error while deleting event in event schedule: ' + error_msg)
                        return error_msg
                }
        }

/**!
setinterval
Changes the interval used to poll the event_schedule table. Default is 60 seconds
**/
        public setinterval(interval::integer) => {
                .'interval' = #interval
        }

/**!
getinterval
Retrieve the interval used for polling the event_schedule table
**/
        public getinterval() => .'interval'

/**!
setdatabase
Set the database where the event_schedule table resides
**/
        public setdatabase(database::string) => {
                .'database' = #database
        }

/**!
events
Will return a map with all events in the event_schedule table
**/
        public events() => {
                if(.'database' -> size) => {
                        local(out = map)
                        inline(
                                -database = .'database',
                                -table = 'event_schedule',
                                -maxrecords = all,
                                -search) =>
                        {
                                records => {
                                        #out -> insert(field('id') = map(
                                                'url' = field('event_url'),
                                                'start' =  field('event_start'),
                                                'end' = field('event_end'),
                                                'delay' = field('event_delay'),
                                                'restart' = field('event_restart'),
                                                'has_username' = (string(field('event_username')) -> size > 0),
                                                'has_password' = (string(field('event_password')) -> size > 0),
                                                'next_run' = field('next_run_datetime'),
                                                'bootup_value' = field('server_bootup_datetime'),
                                        ))
                                }
                        }
                        return #out
                else
                        fail(-1, 'No database set for event_schedule table')
                }
        }

/**!
active_tick
The actual polling method
Will at a set interval check the event_schedule table for urls to run. If found then it uses include_url to call it.
If left unprovoked will do repeat check every minute (60 seconds)
At each run it also checks if it's time to do cleanup. A task done every hour. At cleanup it will delete any events that are not to be run again.
**/
        public active_tick() => {
                if(.'database' -> size) => {
                        local(next_datetime = date)
                        local(compare_date = date)
                        #compare_date -> subtract(-minute = 60)

                        inline(
                                -database = .'database',
                                -table = 'event_schedule',
                                -lte,
                                'next_run_datetime' = date,
                                -maxrecords = all,
                                -search) =>
                        {

                                records => {

                                        // checking if the event is relevant to run
                                        if(field('event_restart') == 'Y' || date(field('server_bootup_datetime')) >= .'initiated') => {

                                                // If either event_username or event_password has values then use them in the call. Values need to be decrypted first
                                                if(field('event_username') -> size > 0 || field('event_password') -> size > 0) => {
                                                        include_url(string(field('event_url')),
                                                                -username = decrypt_blowfish(-seed = .'encrypt_seed', field('event_username') -> decodehex),
                                                                -password = decrypt_blowfish(-seed = .'encrypt_seed', field('event_password') -> decodehex)
                                                        )?null
                                                else
                                                        include_url(string(field('event_url')))?null
                                                }
                                                #next_datetime = date
                                                #next_datetime -> add(-minute = integer(field('event_delay')))
                                                if(integer(field('event_delay')) > 0 && (string(field('event_end')) -> size == 0 || date(field('event_end')) >= #next_datetime)) => {
                                                        // reschedule event
                                                        inline(
                                                                -keyfield = 'id',
                                                                -keyvalue = field('id'),
                                                                'next_run_datetime' = #next_datetime,
                                                                -update) =>
                                                        {
                                                                error_code != 0 ? log_critical('Error while updating event schedule: ' + error_msg)
                                                        }
                                                else(integer(field('event_delay')) == 0 || (string(field('event_end')) -> size != 0 && date(field('event_end')) < #next_datetime))
                                                        // delete the record since it's no longer to be used
                                                        inline(
                                                                -keyfield = 'id',
                                                                -keyvalue = field('id'),
                                                                -delete) =>
                                                        {
                                                                error_code != 0 ? log_critical('Error while deleting event schedule: ' + error_msg)
                                                        }
                                                }
                                        }


                                }
                        }
                        if(#compare_date > .'last_cleanup') => {
                                // would like to use regular inline here but can't make out how to set up the search criteria
                                inline(
                                        -database = 'test',
                                        -table = 'event_schedule',
                                        -sql = 'DELETE FROM event_schedule
WHERE
        (event_end IS NOT NULL AND
        event_end < NOW()) OR
        (event_restart = "N" AND
        server_bootup_datetime < "' + .'initiated' + '")') =>
                                {
                                        error_code != 0 ? log_critical('Error while deleting expired event in event schedule: ' + error_msg)
                                }

                                .'last_cleanup' = date
                        }
                }

                return .'interval'
        } // active_tick

}

event_schedule -> setdatabase('myDB') // change this to whatever suits you

log_critical('event_schedule started up')

?>

#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Brad Lindsay-2
Nice work! Have you submitted it to TagSwap yet?

Brad


On Jul 15, 2011, at 8:46 AM, Jolle Carlestam wrote:

> 14 jul 2011 kl. 15.32 skrev Steve Piercy - Web Site Builder:
>
>>> 14 jul 2011 kl. 15.20 skrev Deco Rior:
>>>
>>>> The event_schedule is gone?
>>>
>>> Yes, but I would say that it's replaced with a better option.
>>> The way to go is to use thread objects instead.
>>
>> Using the better option of thread objects, how would you handle the situation where it is a one-time event that must be scheduled relative to some user interaction, and after the thread object is created, the server restarts or has an ISE?
>>
>> --steve
>
> (Sorry for not replying faster.)
> By creating my oven event scheduler.
> For Lassoers that don't want to do that here's a suggested event_schedule thread.
> It uses a table to store events and will poll the event_schedule table at a given interval. Default is once a minute but the value can be changed by a setting.
> Events are created by the event_schedule -> create method. Example:
> event_schedule -> create('http://example.com',
>                -start = date,
>                -end = null,
>                -delay = 4,
>                -restart = true,
>                -username = string,
>                -password = string,
>                -silent = false // Default is true. If true will return no results from a call
> )
>
> I have refrained from providing a ready made form as I see that more for each developers task.
>
> if username and or password is provided they will be stored encrypted in the table. Simple blowfish but still keep prying eyes from seeing cleartext values.
>
> To check stored events use the method event_schedule -> events. It will return a map of maps with all records in the event_schedule table.
>
> Removing methods before their time is done using event_schedule -> remove('id_value')
>
> There're also some other methods provided. See the code for info.
>
> I scanned the Lasso 8.5 reference for inspiration. http://reference.lassosoft.com/Reference.LassoApp?[Event_Schedule]
>
> To start using it create an event_schedule table in your preferred database. SQL code is supplied in the thread comments
> Place a file with the event_schedule code in LassoStartup and restart Lasso server.
>
> The provided thread should be considered experimental so far. It has not in any way been battle tested. I welcome any suggestions of improvement or bug calls.
> HDB
> Jolle
>
> -----------------------
> Code:
> <?LassoScript
>
> /**!
> event_schedule
> Experimental Event Schedule thread for Lasso 9
> Will at a set interval check the event_schedule table for urls to run. If found then use include_url to call it.
> If left unprovoked will do repeat check every minute (60 seconds).
>
> ### Methods:
> Set the database where the event_schedule table resides
> event_schedule -> setdatabase('test')
>
> Create an event that is stored in the event_schedule table and run at the given interval. If no delay value is provided it will run once. Due to a present bug in Lasso 9 thread handling all params must be provided, even the optional ones.
> event_schedule -> create('http://example.com',
>                -start = date,
>                -end = null,
>                -delay = 4,
>                -restart = true,
>                -username = string,
>                -password = string,
>                -silent = false // Default is true. If true will return no results from a call
> )
>
> Will return a map with all events in the event_schedule table
> event_schedule -> events
>
> Changes the interval used to poll the event_schedule table. Default is 60 seconds
> event_schedule -> setinterval(120) // will change the check param to every two minutes.
>
> Retrieve the interval used for polling the event_schedule table
> event_schedule -> getinterval
>
> Removes the event with the given id from the event_schedule table
> event_schedule -> remove('id_value')
>
> ### SQL for creating the event_schedule table
> CREATE TABLE `event_schedule` (
>  `event_url` varchar(255) CHARACTER SET utf8 COLLATE utf8_swedish_ci DEFAULT NULL COMMENT 'The URL of the page which should be executed at the time set by the other keyword/value parameters.',
>  `event_start` datetime DEFAULT NULL COMMENT 'The date and time at which to execute the event. By default set to the current date and time.',
>  `event_end` datetime DEFAULT NULL COMMENT 'The date and time to stop executing a repeating event.',
>  `event_delay` int(9) DEFAULT NULL COMMENT 'The number of minutes to delay between repetitions of a repeating event.',
>  `event_restart` enum('Y','N') DEFAULT 'N' COMMENT 'If set to true (Y) then the event will be rescheduled after the server restarts. If not set then the event will not be rescheduled. Defaults to False (N).',
>  `event_username` varchar(255) DEFAULT NULL COMMENT 'An optional username which will be used to authenticate the execution of the scheduled event. Stored encrypted in the record.',
>  `event_password` varchar(255) DEFAULT NULL COMMENT 'An optional password which will be used to authenticate execution of the scheduled event. Stored encrypted in the record.',
>  `next_run_datetime` datetime DEFAULT NULL COMMENT 'Set by the system to the next time this event should be run',
>  `server_bootup_datetime` datetime DEFAULT NULL COMMENT 'Date and time for when lasso server was started. Set when the event is created in order to know if an event should be triggered or not in relation to flag set in event_restart. ',
>  `id` bigint(30) NOT NULL AUTO_INCREMENT,
>  PRIMARY KEY (`id`),
>  KEY `next_run_datetime` (`next_run_datetime`)
> ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8
>
> 2011-07-15      JC      First version.
>
> **/
> define event_schedule => thread {
>
>        data
>                public initiated::date, // when the thread was initiated. Most likely at Lasso server startup
>                public database::string, // Database where to find the event_schedule table
>                public interval::integer, // Seconds between each poll of the event_schedule table
>                private last_cleanup::date, // date and time for when the last cleanup was performed
>                private encrypt_seed::string // seed used to encrypt username and password values
>
>        public onCreate() => {
>                .'interval' = 60
>                .'initiated' = date(date -> format('%q')) // need to set format to get rid of nasty hidden fractions of seconds
> //              .'encrypt_seed' = 'AKJHS726KLKJS6823837sgJDIUERHDGJHSGJHJHGJSHHjhsaje84JJJJEWU48' + string(server_ip) + 'ajd2837628' Not working yet, still looking for a replacement to server_name or server_ip
>                .'encrypt_seed' = 'AKJHS726KLKJS6823837sgJDIUERHDGJHSGJHJHGJSHHjhsaje84JJJJEWU48ajd2837628'
>        }
>
> /**!
> create
> Create an event that is stored in the event_schedule table and run at the given interval. If no delay value is provided it will run once. Due to a present bug in Lasso 9 thread handling all params must be provided, even the optional ones.
> **/
>        public create(
>                url::string, // The URL of the page which should be executed at the time set by the other keyword/value parameters.
>                start::date = date, // The date and time at which to execute the event. By default set to the current date and time.
>                end::any = null, // The date and time to stop executing a repeating event.
>                delay::integer = 0, // The number of minutes to delay between repetitions of a repeating event.
>                restart::boolean = false, // If set to true (Y) then the event will be rescheduled after the server restarts. If not set then the event will not be rescheduled. Defaults to False (N).
>                username::string = string, // An optional username which will be used to authenticate the execution of the scheduled event. Stored encrypted in the record.
>                password::string = string, // An optional password which will be used to authenticate execution of the scheduled event. Stored encrypted in the record.
>                silent::boolean = true // If false report feedback from exekution
>        ) => {
>
>                fail_if(.'database' -> size == 0, -1, 'No database has been set')
>                fail_if(#url -> size > 255, -1, 'URL is too long (max 255 chars)')
>                fail_if(!valid_date(#start, -strict), -1, 'Start date is not valid')
>                fail_if(#end != null && !valid_date(#end, -strict), -1, 'End date is not valid')
>                fail_if(#username -> size > 127, -1, 'Username is too long (max 127 chars)')
>                fail_if(#password -> size > 127, -1, 'Password is too long (max 127 chars)')
>
>                inline(
>                        -database = .'database',
>                        -table = 'event_schedule',
>                        'event_url' = #url,
>                        'event_start' = #start,
>                        'event_end' = #end,
>                        'event_delay' = (#delay < 1 ? null | #delay),
>                        'event_restart' = (#restart ? 'Y' | 'N'),
>                        'event_username' = (#username -> size == 0 ? null | encrypt_blowfish(-seed = .'encrypt_seed', #username)->encodehex),
>                        'event_password' = (#password -> size == 0 ? null | encrypt_blowfish(-seed = .'encrypt_seed',#password)->encodehex),
>                        'next_run_datetime' = #start, // Set by the system to the next time this event should be run.
>                        'server_bootup_datetime' = .'initiated', // Date and time for when lasso server was started. Set when the event is created in order to know if an event should be triggered or not in relation to flag set in event_restart.
>                        'id' = lasso_uniqueid,
>                        -add) => {
>
>                        error_code != 0 ? log_critical('Error while creating event in event schedule: ' + error_msg)
>                        !#silent ? return error_msg
>                }
>        }
>
>        public create(
>                url::string,
>                -start::date = date,
>                -end::any = null,
>                -delay::integer = 0,
>                -restart::boolean = false,
>                -username::string = string,
>                -password::string = string,
>                -silent::boolean = true
>        ) => .create(#url, #start, #end, #delay, #restart, #username, #password, #silent)
>
>        public create(
>                url::string,
>                -start::date = date,
>                -end::any = null,
>                -delay::integer = 0,
>                -restart::boolean = false
>        ) => .create(#url, #start, #end, #delay, #restart, string, string, true)
>
> /**!
> remove
> Removes the event with the given id from the event_schedule table
> **/
>        public remove(
>                id::string
>        ) => {
>                fail_if(.'database' -> size == 0, -1, 'No database has been set')
>
>                inline(
>                        -database = .'database',
>                        -table = 'event_schedule',
>                        -keyfield = 'id',
>                        -keyvalue = #id,
>                        -delete) => {
>                        error_code != 0 ? log_critical('Error while deleting event in event schedule: ' + error_msg)
>                        return error_msg
>                }
>        }
>
> /**!
> setinterval
> Changes the interval used to poll the event_schedule table. Default is 60 seconds
> **/
>        public setinterval(interval::integer) => {
>                .'interval' = #interval
>        }
>
> /**!
> getinterval
> Retrieve the interval used for polling the event_schedule table
> **/
>        public getinterval() => .'interval'
>
> /**!
> setdatabase
> Set the database where the event_schedule table resides
> **/
>        public setdatabase(database::string) => {
>                .'database' = #database
>        }
>
> /**!
> events
> Will return a map with all events in the event_schedule table
> **/
>        public events() => {
>                if(.'database' -> size) => {
>                        local(out = map)
>                        inline(
>                                -database = .'database',
>                                -table = 'event_schedule',
>                                -maxrecords = all,
>                                -search) =>
>                        {
>                                records => {
>                                        #out -> insert(field('id') = map(
>                                                'url' = field('event_url'),
>                                                'start' =  field('event_start'),
>                                                'end' = field('event_end'),
>                                                'delay' = field('event_delay'),
>                                                'restart' = field('event_restart'),
>                                                'has_username' = (string(field('event_username')) -> size > 0),
>                                                'has_password' = (string(field('event_password')) -> size > 0),
>                                                'next_run' = field('next_run_datetime'),
>                                                'bootup_value' = field('server_bootup_datetime'),
>                                        ))
>                                }
>                        }
>                        return #out
>                else
>                        fail(-1, 'No database set for event_schedule table')
>                }
>        }
>
> /**!
> active_tick
> The actual polling method
> Will at a set interval check the event_schedule table for urls to run. If found then it uses include_url to call it.
> If left unprovoked will do repeat check every minute (60 seconds)
> At each run it also checks if it's time to do cleanup. A task done every hour. At cleanup it will delete any events that are not to be run again.
> **/
>        public active_tick() => {
>                if(.'database' -> size) => {
>                        local(next_datetime = date)
>                        local(compare_date = date)
>                        #compare_date -> subtract(-minute = 60)
>
>                        inline(
>                                -database = .'database',
>                                -table = 'event_schedule',
>                                -lte,
>                                'next_run_datetime' = date,
>                                -maxrecords = all,
>                                -search) =>
>                        {
>
>                                records => {
>
>                                        // checking if the event is relevant to run
>                                        if(field('event_restart') == 'Y' || date(field('server_bootup_datetime')) >= .'initiated') => {
>
>                                                // If either event_username or event_password has values then use them in the call. Values need to be decrypted first
>                                                if(field('event_username') -> size > 0 || field('event_password') -> size > 0) => {
>                                                        include_url(string(field('event_url')),
>                                                                -username = decrypt_blowfish(-seed = .'encrypt_seed', field('event_username') -> decodehex),
>                                                                -password = decrypt_blowfish(-seed = .'encrypt_seed', field('event_password') -> decodehex)
>                                                        )?null
>                                                else
>                                                        include_url(string(field('event_url')))?null
>                                                }
>                                                #next_datetime = date
>                                                #next_datetime -> add(-minute = integer(field('event_delay')))
>                                                if(integer(field('event_delay')) > 0 && (string(field('event_end')) -> size == 0 || date(field('event_end')) >= #next_datetime)) => {
>                                                        // reschedule event
>                                                        inline(
>                                                                -keyfield = 'id',
>                                                                -keyvalue = field('id'),
>                                                                'next_run_datetime' = #next_datetime,
>                                                                -update) =>
>                                                        {
>                                                                error_code != 0 ? log_critical('Error while updating event schedule: ' + error_msg)
>                                                        }
>                                                else(integer(field('event_delay')) == 0 || (string(field('event_end')) -> size != 0 && date(field('event_end')) < #next_datetime))
>                                                        // delete the record since it's no longer to be used
>                                                        inline(
>                                                                -keyfield = 'id',
>                                                                -keyvalue = field('id'),
>                                                                -delete) =>
>                                                        {
>                                                                error_code != 0 ? log_critical('Error while deleting event schedule: ' + error_msg)
>                                                        }
>                                                }
>                                        }
>
>
>                                }
>                        }
>                        if(#compare_date > .'last_cleanup') => {
>                                // would like to use regular inline here but can't make out how to set up the search criteria
>                                inline(
>                                        -database = 'test',
>                                        -table = 'event_schedule',
>                                        -sql = 'DELETE FROM event_schedule
> WHERE
>        (event_end IS NOT NULL AND
>        event_end < NOW()) OR
>        (event_restart = "N" AND
>        server_bootup_datetime < "' + .'initiated' + '")') =>
>                                {
>                                        error_code != 0 ? log_critical('Error while deleting expired event in event schedule: ' + error_msg)
>                                }
>
>                                .'last_cleanup' = date
>                        }
>                }
>
>                return .'interval'
>        } // active_tick
>
> }
>
> event_schedule -> setdatabase('myDB') // change this to whatever suits you
>
> log_critical('event_schedule started up')
>
> ?>
>
> #############################################################
> This message is sent to you because you are subscribed to
>  the mailing list <[hidden email]>.
> To unsubscribe, E-mail to: <[hidden email]>
> To switch to the DIGEST mode, E-mail to <[hidden email]>
> To switch to the INDEX mode, E-mail to <[hidden email]>
> Send administrative queries to  <[hidden email]>
>


#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Jolle Carlestam-3
Thanks!
Not on tag swap yet. Had to leave the computer for a while.
Also keen on improvement suggestions or pointers to possible flaws in the implementation first.

HDB
Jolle

Sent from a thin, flat, touchy device from an undetermined place in space.

15 jul 2011 kl. 14:57 skrev "Brad Lindsay" <[hidden email]>:

> Nice work! Have you submitted it to TagSwap yet?
>
> Brad

#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

stevepiercy
In reply to this post by Jolle Carlestam-3
So you essentially took what existed in pre-9 (and was removed
in 9), and made some modifications.  Your efforts to replace a
very useful and important feature are commendable, and I hope
that LS sees the value of incorporating your efforts back into 9.

Nonetheless I think that the event scheduler in pre-9 is better
because it also had a web-based interface for management,
including the ability to stop and start the event queue.  I
don't know what method pre-9 used to check the queue every
minute, but regardless of the shiny new thread-based method in
9, the lack of this built-in queue is a step backward and a
hurdle to migration of legacy solutions.

--steve


On 7/15/11 at 12:46 PM, [hidden email] (Jolle
Carlestam) pronounced:

>14 jul 2011 kl. 15.32 skrev Steve Piercy - Web Site Builder:
>
>>> 14 jul 2011 kl. 15.20 skrev Deco Rior:
>>>
>>>> The event_schedule is gone?
>>>
>>> Yes, but I would say that it's replaced with a better option.
>>> The way to go is to use thread objects instead.
>>
>>Using the better option of thread objects, how would you handle the situation
>where it is a one-time event that must be scheduled relative to
>some user interaction, and after the thread object is created,
>the server restarts or has an ISE?
>>
>>--steve
>
>(Sorry for not replying faster.)
>By creating my oven event scheduler.
>For Lassoers that don't want to do that here's a suggested event_schedule thread.
>It uses a table to store events and will poll the
>event_schedule table at a given interval. Default is once a
>minute but the value can be changed by a setting.
>Events are created by the event_schedule -> create method. Example:
>event_schedule -> create('http://example.com',
>-start = date,
>-end = null,
>-delay = 4,
>-restart = true,
>-username = string,
>-password = string,
>-silent = false // Default is true. If true will return no
>results from a call
>)
>
>I have refrained from providing a ready made form as I see that
>more for each developers task.
>
>if username and or password is provided they will be stored
>encrypted in the table. Simple blowfish but still keep prying
>eyes from seeing cleartext values.
>
>To check stored events use the method event_schedule -> events.
>It will return a map of maps with all records in the
>event_schedule table.
>
>Removing methods before their time is done using event_schedule
>-> remove('id_value')
>
>There're also some other methods provided. See the code for info.
>
>I scanned the Lasso 8.5 reference for inspiration. http://reference.lassosoft.com/Reference.LassoApp?[Event_Schedule]
>
>To start using it create an event_schedule table in your
>preferred database. SQL code is supplied in the thread comments
>Place a file with the event_schedule code in LassoStartup and restart Lasso server.
>
>The provided thread should be considered experimental so far.
>It has not in any way been battle tested. I welcome any
>suggestions of improvement or bug calls.
>HDB
>Jolle
>
>-----------------------
>Code:
><?LassoScript
>
>/**!
>event_schedule
>Experimental Event Schedule thread for Lasso 9
>Will at a set interval check the event_schedule table for urls
>to run. If found then use include_url to call it.
>If left unprovoked will do repeat check every minute (60 seconds).
>
>### Methods:
>Set the database where the event_schedule table resides
>event_schedule -> setdatabase('test')
>
>Create an event that is stored in the event_schedule table and
>run at the given interval. If no delay value is provided it
>will run once. Due to a present bug in Lasso 9 thread handling
>all params must be provided, even the optional ones.
>event_schedule -> create('http://example.com',
>-start = date,
>-end = null,
>-delay = 4,
>-restart = true,
>-username = string,
>-password = string,
>-silent = false // Default is true. If true will return no
>results from a call
>)
>
>Will return a map with all events in the event_schedule table
>event_schedule -> events
>
>Changes the interval used to poll the event_schedule table. Default is 60 seconds
>event_schedule -> setinterval(120) // will change the check
>param to every two minutes.
>
>Retrieve the interval used for polling the event_schedule table
>event_schedule -> getinterval
>
>Removes the event with the given id from the event_schedule table
>event_schedule -> remove('id_value')
>
>### SQL for creating the event_schedule table
>CREATE TABLE `event_schedule` (
>`event_url` varchar(255) CHARACTER SET utf8 COLLATE
>utf8_swedish_ci DEFAULT NULL COMMENT 'The URL of the page which
>should be executed at the time set by the other keyword/value parameters.',
>`event_start` datetime DEFAULT NULL COMMENT 'The date and time
>at which to execute the event. By default set to the current
>date and time.',
>`event_end` datetime DEFAULT NULL COMMENT 'The date and time to
>stop executing a repeating event.',
>`event_delay` int(9) DEFAULT NULL COMMENT 'The number of
>minutes to delay between repetitions of a repeating event.',
>`event_restart` enum('Y','N') DEFAULT 'N' COMMENT 'If set to
>true (Y) then the event will be rescheduled after the server
>restarts. If not set then the event will not be rescheduled.
>Defaults to False (N).',
>`event_username` varchar(255) DEFAULT NULL COMMENT 'An optional
>username which will be used to authenticate the execution of
>the scheduled event. Stored encrypted in the record.',
>`event_password` varchar(255) DEFAULT NULL COMMENT 'An optional
>password which will be used to authenticate execution of the
>scheduled event. Stored encrypted in the record.',
>`next_run_datetime` datetime DEFAULT NULL COMMENT 'Set by the
>system to the next time this event should be run',
>`server_bootup_datetime` datetime DEFAULT NULL COMMENT 'Date
>and time for when lasso server was started. Set when the event
>is created in order to know if an event should be triggered or
>not in relation to flag set in event_restart. ',
>`id` bigint(30) NOT NULL AUTO_INCREMENT,
>PRIMARY KEY (`id`),
>KEY `next_run_datetime` (`next_run_datetime`)
>) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8
>
>2011-07-15      JC      First version.
>
>**/
>define event_schedule => thread {
>
>data
>public initiated::date, // when the thread was initiated. Most
>likely at Lasso server startup
>public database::string, // Database where to find the
>event_schedule table
>public interval::integer, // Seconds between each poll of the
>event_schedule table
>private last_cleanup::date, // date and time for when the last
>cleanup was performed
>private encrypt_seed::string // seed used to encrypt username
>and password values
>
>public onCreate() => {
>.'interval' = 60
>.'initiated' = date(date -> format('%q')) // need to set format
>to get rid of nasty hidden fractions of seconds
>//              .'encrypt_seed' =
>'AKJHS726KLKJS6823837sgJDIUERHDGJHSGJHJHGJSHHjhsaje84JJJJEWU48'
>+ string(server_ip) + 'ajd2837628' Not working yet, still
>looking for a replacement to server_name or server_ip
>.'encrypt_seed' = 'AKJHS726KLKJS6823837sgJDIUERHDGJHSGJHJHGJSHHjhsaje84JJJJEWU48ajd2837628'
>}
>
>/**!
>create
>Create an event that is stored in the event_schedule table and
>run at the given interval. If no delay value is provided it
>will run once. Due to a present bug in Lasso 9 thread handling
>all params must be provided, even the optional ones.
>**/
>public create(
>url::string, // The URL of the page which should be executed at
>the time set by the other keyword/value parameters.
>start::date = date, // The date and time at which to execute
>the event. By default set to the current date and time.
>end::any = null, // The date and time to stop executing a
>repeating event.
>delay::integer = 0, // The number of minutes to delay between
>repetitions of a repeating event.
>restart::boolean = false, // If set to true (Y) then the event
>will be rescheduled after the server restarts. If not set then
>the event will not be rescheduled. Defaults to False (N).
>username::string = string, // An optional username which will
>be used to authenticate the execution of the scheduled event.
>Stored encrypted in the record.
>password::string = string, // An optional password which will
>be used to authenticate execution of the scheduled event.
>Stored encrypted in the record.
>silent::boolean = true // If false report feedback from exekution
>) => {
>
>fail_if(.'database' -> size == 0, -1, 'No database has been set')
>fail_if(#url -> size > 255, -1, 'URL is too long (max 255 chars)')
>fail_if(!valid_date(#start, -strict), -1, 'Start date is not valid')
>fail_if(#end != null && !valid_date(#end, -strict), -1, 'End
>date is not valid')
>fail_if(#username -> size > 127, -1, 'Username is too long (max
>127 chars)')
>fail_if(#password -> size > 127, -1, 'Password is too long (max
>127 chars)')
>
>inline(
>-database = .'database',
>-table = 'event_schedule',
>'event_url' = #url,
>'event_start' = #start,
>'event_end' = #end,
>'event_delay' = (#delay < 1 ? null | #delay),
>'event_restart' = (#restart ? 'Y' | 'N'),
>'event_username' = (#username -> size == 0 ? null |
>encrypt_blowfish(-seed = .'encrypt_seed', #username)->encodehex),
>'event_password' = (#password -> size == 0 ? null |
>encrypt_blowfish(-seed = .'encrypt_seed',#password)->encodehex),
>'next_run_datetime' = #start, // Set by the system to the next
>time this event should be run.
>'server_bootup_datetime' = .'initiated', // Date and time for
>when lasso server was started. Set when the event is created in
>order to know if an event should be triggered or not in
>relation to flag set in event_restart.
>'id' = lasso_uniqueid,
>-add) => {
>
>error_code != 0 ? log_critical('Error while creating event in
>event schedule: ' + error_msg)
>!#silent ? return error_msg
>}
>}
>
>public create(
>url::string,
>-start::date = date,
>-end::any = null,
>-delay::integer = 0,
>-restart::boolean = false,
>-username::string = string,
>-password::string = string,
>-silent::boolean = true
>) => .create(#url, #start, #end, #delay, #restart, #username,
>#password, #silent)
>
>public create(
>url::string,
>-start::date = date,
>-end::any = null,
>-delay::integer = 0,
>-restart::boolean = false
>) => .create(#url, #start, #end, #delay, #restart, string, string, true)
>
>/**!
>remove
>Removes the event with the given id from the event_schedule table
>**/
>public remove(
>id::string
>) => {
>fail_if(.'database' -> size == 0, -1, 'No database has been set')
>
>inline(
>-database = .'database',
>-table = 'event_schedule',
>-keyfield = 'id',
>-keyvalue = #id,
>-delete) => {
>error_code != 0 ? log_critical('Error while deleting event in
>event schedule: ' + error_msg)
>return error_msg
>}
>}
>
>/**!
>setinterval
>Changes the interval used to poll the event_schedule table. Default is 60 seconds
>**/
>public setinterval(interval::integer) => {
>.'interval' = #interval
>}
>
>/**!
>getinterval
>Retrieve the interval used for polling the event_schedule table
>**/
>public getinterval() => .'interval'
>
>/**!
>setdatabase
>Set the database where the event_schedule table resides
>**/
>public setdatabase(database::string) => {
>.'database' = #database
>}
>
>/**!
>events
>Will return a map with all events in the event_schedule table
>**/
>public events() => {
>if(.'database' -> size) => {
>local(out = map)
>inline(
>-database = .'database',
>-table = 'event_schedule',
>-maxrecords = all,
>-search) =>
>{
>records => {
>#out -> insert(field('id') = map(
>'url' = field('event_url'),
>'start' =  field('event_start'),
>'end' = field('event_end'),
>'delay' = field('event_delay'),
>'restart' = field('event_restart'),
>'has_username' = (string(field('event_username')) -> size > 0),
>'has_password' = (string(field('event_password')) -> size > 0),
>'next_run' = field('next_run_datetime'),
>'bootup_value' = field('server_bootup_datetime'),
>))
>}
>}
>return #out
>else
>fail(-1, 'No database set for event_schedule table')
>}
>}
>
>/**!
>active_tick
>The actual polling method
>Will at a set interval check the event_schedule table for urls
>to run. If found then it uses include_url to call it.
>If left unprovoked will do repeat check every minute (60 seconds)
>At each run it also checks if it's time to do cleanup. A task
>done every hour. At cleanup it will delete any events that are
>not to be run again.
>**/
>public active_tick() => {
>if(.'database' -> size) => {
>local(next_datetime = date)
>local(compare_date = date)
>#compare_date -> subtract(-minute = 60)
>
>inline(
>-database = .'database',
>-table = 'event_schedule',
>-lte,
>'next_run_datetime' = date,
>-maxrecords = all,
>-search) =>
>{
>
>records => {
>
>// checking if the event is relevant to run
>if(field('event_restart') == 'Y' ||
>date(field('server_bootup_datetime')) >= .'initiated') => {
>
>// If either event_username or event_password has values then
>use them in the call. Values need to be decrypted first
>if(field('event_username') -> size > 0 ||
>field('event_password') -> size > 0) => {
>
>include_url(string(field('event_url')),
>-username = decrypt_blowfish(-seed = .'encrypt_seed',
>field('event_username') -> decodehex),
>-password = decrypt_blowfish(-seed = .'encrypt_seed',
>field('event_password') -> decodehex)
>)?null
>else
>
>include_url(string(field('event_url')))?null
>}
>#next_datetime = date
>#next_datetime -> add(-minute = integer(field('event_delay')))
>if(integer(field('event_delay')) > 0 &&
>(string(field('event_end')) -> size == 0 ||
>date(field('event_end')) >= #next_datetime)) => {
>// reschedule event
>inline(
>-keyfield = 'id',
>-keyvalue = field('id'),
>'next_run_datetime' = #next_datetime,
>-update) =>
>{
>error_code != 0 ? log_critical('Error while updating event
>schedule: ' + error_msg)
>}
>else(integer(field('event_delay')) == 0 ||
>(string(field('event_end')) -> size != 0 &&
>date(field('event_end')) < #next_datetime))
>// delete the record since it's no longer to be used
>inline(
>-keyfield = 'id',
>-keyvalue = field('id'),
>-delete) =>
>{
>error_code != 0 ? log_critical('Error while deleting event
>schedule: ' + error_msg)
>}
>}
>}
>
>
>}
>}
>if(#compare_date > .'last_cleanup') => {
>// would like to use regular inline here but can't make out how
>to set up the search criteria
>inline(
>-database = 'test',
>-table = 'event_schedule',
>-sql = 'DELETE FROM event_schedule
>WHERE
>(event_end IS NOT NULL AND
>event_end < NOW()) OR
>(event_restart = "N" AND
>server_bootup_datetime < "' + .'initiated' + '")') =>
>{
>error_code != 0 ? log_critical('Error while deleting expired
>event in event schedule: ' + error_msg)
>}
>
>.'last_cleanup' = date
>}
>}
>
>return .'interval'
>} // active_tick
>
>}
>
>event_schedule -> setdatabase('myDB') // change this to whatever suits you
>
>log_critical('event_schedule started up')
>
>?>
>
>#############################################################
>This message is sent to you because you are subscribed to
>the mailing list <[hidden email]>.
>To unsubscribe, E-mail to: <[hidden email]>
>To switch to the DIGEST mode, E-mail to <[hidden email]>
>To switch to the INDEX mode, E-mail to <[hidden email]>
>Send administrative queries to  <[hidden email]>
>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
-- --
Steve Piercy               Web Site Builder              
Soquel, CA
<[hidden email]>                  <http://www.StevePiercy.com/>


#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Brad Lindsay-2
In reply to this post by Jolle Carlestam-3
On Jul 15, 2011, at 9:21 AM, Jolle Carlestam wrote:
> Thanks!
> Not on tag swap yet. Had to leave the computer for a while.
> Also keen on improvement suggestions or pointers to possible flaws in the implementation first.

If you're interested, we could post it to Github and work on making a LassoApp out of it to give it a web-based interface. I could find some time to help work on that.


Brad
#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Advice on automatic removal of files

Jolle Carlestam-3
15 jul 2011 kl. 16:12 skrev "Brad Lindsay" <[hidden email]>:

> On Jul 15, 2011, at 9:21 AM, Jolle Carlestam wrote:
>> Thanks!
>> Not on tag swap yet. Had to leave the computer for a while.
>> Also keen on improvement suggestions or pointers to possible flaws in the implementation first.
>
> If you're interested, we could post it to Github and work on making a LassoApp out of it to give it a web-based interface. I could find some time to help work on that.
>
>
> Brad

That sounds like a really interesting idea!
I have never made a lassoapp before!

HDB
Jolle

Sent from a thin, flat, touchy device from an undetermined place in space.


#############################################################
This message is sent to you because you are subscribed to
  the mailing list <[hidden email]>.
To unsubscribe, E-mail to: <[hidden email]>
To switch to the DIGEST mode, E-mail to <[hidden email]>
To switch to the INDEX mode, E-mail to <[hidden email]>
Send administrative queries to  <[hidden email]>