Forum Moderators: coopster & phranque

Message Too Old, No Replies

Including an external file that MIGHT have errors

         

csdude55

12:11 am on Sep 1, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've gotten in the habit of moving anything that might be changed regularly to a separate file; eg, spam filters in the contact form. This way I don't have to modify the main script, and if I accidentally code an error then it doesn't kill the whole script.

I've been including them for years using:

eval { require '/path/to/whatever.filter' };


Is there a better way to include them, though, and NOT die if there's an error in the included file?

I tried adding Carp to whatever.filter, but it didn't do anything:

use Carp;

function whatever {
carp "oops";

# do filters and stuff
return $foo;
}
1;

lucy24

1:47 am on Sep 1, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



and NOT die if there's an error in the included file
That would kinda defeat the point of REQUIRE, though, wouldn't it.

How many included files are potentially involved? Is it safe to assume there are lots and lots of them, so putting an error-handling routine in each file-to-be-included isn't a workable option? (And still wouldn't help if one of those files-to-be-included eloped with a gremlin and went mysteriously walkabout...)

I'm trying to think if there would be a viable way to make a single intermediary function for all includes. So in place of include/require/whatever, it would be a detour to {get this file if you can, and otherwise return some kind of placeholder that won't crash the originationg function} where the filename becomes something like a variable passed to {hypothetical mystery function}.

csdude55

3:57 am on Sep 1, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



That would kinda defeat the point of REQUIRE, though, wouldn't it.

It would, which is why I was hoping there might be an alternative. PHP's include doesn't throw a fatal error like require does, but I don't think there's anything like that in Perl :-(

At the very least, a return value when you require something would be great; eg,

$check = require '/path/to/whatever.filter';

if ($check) {
# do whatever
}


I did some research and THOUGHT that using use and changing the .filter to a .pm would work, but it still throws the same error:

# main script
use lib '/path/to';
use whatever;

$bar = whatever('example');

# /path/to/whatever.pm
package whatever;
use Carp;

sub whatever {
carp "oops";

// do filters and stuff
return $foo;
}
1;


How many included files are potentially involved?

I have 12 potential files that could be included so it's not impossible to add an error-handling routine. What do you suggest?

The problem I have is that they all have a series of conditions and regexes that I regularly update, but every once in awhile I might make a typo. I use eval { } so that I don't wake up to realize that the script has been dead for 8 hours, but of course 12 of those slows it down unnecessarily.

I know how to check to see if /path/to/whatever.filter exists, of course, I just don't know how to make sure that it's error-free without actually running it.

csdude55

4:32 am on Sep 1, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Update; I found an alternative that seems to work:

$result = do '/path/to/whatever.filter';

if ($result) {
require '/path/to/whatever.filter';
$check = defined(&whatever);
}

if ($check) {
# do whatever
}


What do you think, is do actually any faster than eval?

lucy24

5:29 am on Sep 1, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Heh. You may have forgotten that I don't speak perl--not at all, not even the three words or so of php that I can dredge up. So in questions of this sort, I can only think in general concepts: Does Language X have a structure that would let you go through Logical Process Y?

Speed is always a tricky question. Every added test or verification contributes its mite to the time taken. And it all seems like wasted time--until the day that something fails and, because you didn't take the time to verify, everything goes crashing to a halt. It's like buying travel insurance, or paying a mechanic to look at a car you're thinking of buying. Each time that nothing goes wrong, it's money down the drain ... until the day something does go wrong and then the investment is justified.

robzilla

8:46 am on Sep 1, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The fastest way would be to simply validate your Perl files before they go live. Right now, you're burdening your script with the chance that one of those required files may fail due to a coding error, and so it has to check for that every time the script is called upon (essentially executing the code in the included file twice, which is obviously less efficient). Validate beforehand and you can skip that step.

csdude55

6:15 pm on Sep 1, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Very true, @robzilla, but I know me :-/ I do the wide majority of these updates between 2 and 4am, and there have been more than a few times that this double-check saved my butt! LOL 8-9 hours of downtime can mean a loss of $75-100, not to mention long term loss of new users. So it's like @lucy24 said: it's a waste of time, until it's not. I'd rather have a double-check in place than risk it.

I was hoping that there was a faster way to double check, but I guess not :-(

I did a benchtest on jdoodle.com, though, and found that DO is marginally faster on a FAIL in this scenario. I tested using 'random.lib', which (I assume) doesn't exist on jdoodle, so if it did exist then it might be faster.

BUT! When the test succeeds, the DO has a second REQUIRE, while the EVAL does not. So since I expect a success considerably more often than a fail, the combination of the DO and REQUIRE would probably end up being slower than the single EVAL.

So for my purposes, I'm going to stick with EVAL unless someone can suggest a third option.

It's also notable that when I did a simple math equation (eg, do '$x = 14 * 12'), EVAL was MUCH faster! Like 100 times faster.

Here's my test:

# using Perl v. 5.22.0
use Time::HiRes qw(gettimeofday);

# Test 1: 0.196840047836304
$start_time = microtime(TRUE);

for ($i = 0; $i < 10000; $i++) {
do 'random.lib';
}

$end_time = microtime(TRUE);

print 'Test 1: ';
print ($end_time - $start_time);
print "\n";

# Test 2: 0.254052877426147
$start_time = microtime(TRUE);

for ($i = 0; $i < 10000; $i++) {
eval { require 'random.lib' };
}

$end_time = microtime(TRUE);

print 'Test 2: ';
print ($end_time - $start_time);


I have a microtime() function that I borrowed from somewhere that's in the script, too. I didn't post it for the sake of brevity, but I can upon request.

csdude55

7:33 pm on Sep 1, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Since we're having fun... I THINK that putting EVAL inside of the included file will be faster, although it leaves an opportunity for an error.

This is how I did it:

# main script
require '/path/to/whatever.filter';
$check = defined(&whatever);

if ($check) {
$bar = whatever('example');
}

# /path/to/whatever.filter
sub whatever {
$foo = false;

eval '
# do stuff
$foo = true;
';

return $foo;
}
1;


For my test I'm intentionally putting a syntax error in the do stuff section.

Using the same benchtest script that I posted earlier, my results are:

With eval ' require... ' in the main script:
0.517141103744507

With eval '...'; in the filter:
0.126014947891235

My tests aren't identical, though, since eval ' require "/path/to/whatever.filter" '; will fail in the test while the second one actually runs. But I would think that that would make the require run FASTER, not slower, which would mean that having EVAL in the filter would be even faster than I think.

If I use eval { ... }; instead of eval ' ... '; in the filter, through, it throws an error. Apparently the { } makes EVAL run at compile-time, while ' ' makes it run at run-time:

[perldoc.perl.org...]

That leaves an unfortunate potential error if I use a single-quote within the EVAL and don't escape it, though, even if it's in a comment.

Of course, it also leaves room for a fatal error if I accidentally delete /path/to/whatever.filter altogether (or it's just not found for whatever reason), but in my case I think that would be an extremely rare problem.

lucy24

12:25 am on Sep 2, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I THINK that putting EVAL inside of the included file will be faster, although it leaves an opportunity for an error.
At this point you could also start thinking about the likelihood of various types of error. If the file-to-be-included doesn't exist at all, it would be a pretty lethal error--but not one that is awfully likely to occur, unless your .pl files really do have a habit of eloping with gremlins. Even if you did all this at 3 AM before crashing for the night, you can manage one quick test-run just to ensure you haven't, say, mistyped the name of the file. And then if the included file itself contains error handling, there should be a limit to how drastic the consequences of an error might be.

robzilla

8:39 am on Sep 2, 2021 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Even if you did all this at 3 AM before crashing for the night, you can manage one quick test-run just to ensure you haven't, say, mistyped the name of the file.

Exactly. You could write a simple test script and run that after making your edits, or just run perl -c yourscript.pl when you're done with a file.

What you're writing now is very forgiving, perhaps a little too much. If there's an error, will you even know it?

Brett_Tabke

12:22 pm on Sep 2, 2021 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



My editor (editplus) has 'cmds' you can define. One of my defined ops is to do a perl -cw check on the file. So I do a syntax check every time I save.

That's great - but running winders on the desktop and *nix on the server leads to some occasion issues with modules, but it mostly works.

> require twice

I thought require only actually executed the require once? I remember there are ways to get around that ...

[docstore.mik.ua...]