That rule is too complicated. Assuming that the requested URL-path actually contains the literal string "(null)", then a one-line rule should suffice for all the cases your rule covers:

RewriteRule \(null\) - [F] 

Since this pattern is un-anchored, it will deny requests with "(null)" occurring anywhere in the requested
URL-path; you don't need those extra RewriteConds with leading and trailing ".*" subpatterns at all.

However, I suspect that your theory is correct: The requested URL-path probably does not contain the literal string "(null)" and it's appearance in your access log is indicative of something else, such as a literal null character (encoded value %00) in the URL-path.

I'm not sure how you'd detect that; I've never seen this problem before. I suppose you could try making a filter by specifying what you will accept and negating that test, rather than specifying what you want to reject. Also, look at THE_REQUEST, so that you are examining the client request before any un-encoding, un-escaping, or any other 'character transforms' are applied. Something like:

RewriteCond %{THE_REQUEST} !^[A-Z]+\ /[0-9A-Za-z\-._#?&%]+\ HTTP/
RewriteRule .* - [F]

Or look for a singly- or multiply-encoded Null character:

RewriteCond %{THE_REQUEST} \%(25)*00
RewriteRule .* - [F]

You may have to tweak the pattern on my second proposed rule (of the three) to include all the characters you allow in your URL-paths and query strings; I just took a guess.

Trident/4.0 is the new page-rendering engine used in IE8, analogous to the "Gecko" rendering engine as used in Mozilla-based browsers. It is not any kind of "scanner." However, since Microsoft can't seem to issue .NET CLR updates fast enough to fill out the MSIE user-agent string to a more-satisfying 2048 bytes, they are now finding it necessary to "brag" about the new Trident release in the UA string. (This comment made with tongue firmly in cheek.)

Jim

Thanks for Yet Another prompt and informative reply, Jim. I'll fiddle around with your Null-character code because that fits with what I'm seeing -- or what Apache thinks it's seeing, but isn't.

: )

And about Trident's IE8 connection, that's interesting news and suggestive that this /(null) thing might be the tip of a nasty iceberg. (Akin to when the ";1813" AVG Toolbar screw-up [webmasterworld.com] was first foisted upon the world and a curious trickle turned into a flood. I still see ;1813s.)

Are you seeing any /(null)s yet?

Btw, I thought Trident was photographic drum scanner (not file scanner) software because of Googled results like this [colorbytesoftware.com]. Wow. I wonder if ColorByte Software is prepared to tackle intellectual property issues given MS's product ID similarity...

Thanks again!

P.S.

This --

RewriteCond %{REQUEST_URI} \(null\) - [F]

-- generated the dreaded: "RewriteCond: bad flag delimiters."

But this --

RewriteCond %{THE_REQUEST} \%(25)*00

RewriteRule .* - [F]

-- is A-OK, ready to shine as the newly-named nullKill:)

Those requests aren't actual null characters.

My guess is these requests are caused by javascript errors, probably from a poorly scripted toolbar.

This --

 RewriteCond %{REQUEST_URI} \(null\) - [F] 

-- generated the dreaded: "RewriteCond: bad flag delimiters."

Well, you got off easy with just a "bad flag delimeters" error -- That's a *really* invalid directive, a cross between a RewriteCond and a RewriteRule, apparently... :o

But this --

RewriteCond %{THE_REQUEST} \%(25)*00
RewriteRule .* - [F]

-- is A-OK, ready to shine as the newly-named nullKill :)

Are you seeing any /(null)s yet?

I've never ever seen a request for /(null), so that's why I was "guessing" about the solution above. However, my sites would block it, since %00 doesn't correspond to a URI-valid character when decoded.

Jim

Jim: Well, another one just came a'calling, to the tune of a 404. So we can rule out "\%(25)*00", alas. Back to the artful/creative coding board for us!

How do your sites [F] this, versus sending a 404?

All: The next time anyone's procrastinating some tedious chore, could you kindly grep your largest 2009 access_log and/or error_log just to see if any /(null) calls have crossed your path? My server+traffic are small compared to most pros here, but my /(null) count is now multiplying exponentially.

At least whatever-it-is isn't at Michael Crichton's "Andromeda" growth-level yet. Yet...

> How do your sites [F] this, versus sending a 404?

Well, I don't know that my sites *would* 403 this -- As I said, I've never seen a /(null) request.

However, they would 403 a request that had %(25)* followed by any numbers from 00 to 1F (hexadecimal) inclusive, or any numbers above 7E hexadecimal. Encoded characters in those ranges do not decode to printable characters, and therefore will never appear in a valid URL.

Since I do not know how to block /(null) requests, I suggested three rules. I still do. The first rule I suggested may indeed be the one you want, but it needs to be used exactly as I typed it above. It was quite evidently mis-copied and therefore caused a "bad flag delimiters" error.

Put all three rules up, and see if these /(null) requests still get through. Then comment them out one a time to see which one is (or which ones are) actually blocking anything.

> At least whatever-it-is isn't at Michael Crichton's "Andromeda" growth-level yet. Yet...

I hope it doesn't get to that level. I've found that unlike Crichton's virus, crying won't help. :( Ya gotta slap 'em with a one-way ticket on route 403.

Jim

I remember getting 404s on requests for /0 (zero) and /defaultsite I believe.

It turned that was the part of my drop-down navigation created by OpenCube software that by default puts JavaScript links with "0" (zero) and "void" value. Once you update all links for your site, the inactive one still stays (like zero one) - until you fix it.

Anyhow, my two cents are that this could be caused by some JavaScipt or similar related software, toolbar, navigation solution you use, things like Google Analytics or AddThis, and so on.

Add me to the list of those receiving these. I started a topic elsewhere that addresses this same issue. I've asked that it be locked and I'll add to this topic. I'm on Windows though and we're in the Apache forum. Be nice to me! :)

I've been seeing these all day, every day since IE8 launched in Beta. At first I thought someone was probing for something, I'm new to this logfile watching stuff. Then I realized the consistency in the UAs and inconsistency in IPs. I'd say a browser bug of some sort? Whatever it is, they are getting annoying and I'm waiting for me dev to come online to see what we can do to address them.

Anyhow, my two cents are that this could be caused by some JavaScipt or similar related software, toolbar, navigation solution you use, things like Google Analytics or AddThis, and so on.

That seems to be the consensus of my dev too, a JavaScript error of some sort. We're setting up a custom page to alert users of IE8 Beta about the problem. This for the ISAPI_Rewrite 2.0 httpd file on Windows...

#IE8 Trident/4.0 (null)
RewriteRule .*\(null\) /errors/null.asp [I,L]

It turns out that the string sent by the client --at least in some documented cases reported to me-- is %28null%29

In other words, the parentheses are hex-encoded.

So, we might want to use:

RewriteCond %{THE_REQUEST} \%(25)*28null\%(25)*29 [NC]
RewriteRule .* - [F]

Again, the (25)* is to allow for double-encoded or multiply-encoded requests. So we might see
%28null%29 or %2528null%2529 or %252528null%252529, depending on how many encoding nodes the request passes through.

Note that I don't know where these requests are coming from or whether it's a good idea to 403 them. I'm only trying to figure out a way to detect them, and I'm making no specific recommendations on the disposition of these requests at this time.

Jim

It turns out that the string sent by the client --at least in some documented cases reported to me-- is %28null%29

I haven't seen any of those requests. I'm not saying that those urls were not requested that way, but I'll put forward the possibility that those requests were hex encoded by the logging software for security reasons.

Jim,

Key_Master is correct, the request that I saw was for http://www.example.com/(null) My reporting software escaped those characters.

As suddenly as it began over here, the curious "/(null)" oddity appears to have stopped. No requests today at all. (knocks wood)

FWIW, yesterday afternoon I decided not to further mess with my already too-flabby mod_rewrite code and I put up an .html file that's blank but for some no-cache METAs and a 1x1 IMG SRC .jpg in the body. I also lopped off the extension so the 446-byte file appears in my top level dir as:

null

I also figured what the heck and added a no-META, no-IMG 63-byte file literally titled (null) -- no ASCII parens, no nuttin' -- so I could note the sizes of which, if any, files were retrieved instead of my custom 404 page.

Result?

My access_log (ELF) shows that all requests for "/(null)" successfully retrieved the 446-byte file listed as "null". (As surmised, there were no requests for jpg; ditto any other files during any session.) And "MSIE 8.0" and "Trident/4.0" (& variously numbered .NET CLRs) were still the only commonalities.

You might want to check the server response headers for those null/(null) requests. Because the file has no extension, the server may not serve it with a Content-Type of text/html. If that is the case, you'd either want to add a file extension and an internal rewrite from the null/(null) URLs to that file, or change the content of the file to match the Content-Type that the server does return for an extensionless file -- text/plain perhaps.

Just keep in mind that the Content-Type header info is generally derived from the file extension on the server.

Jim

1.) Well, I spoke too soon. They're baaaack, with variations:

One host's request for top-level /(null) resulted in their getting my 69-byte test file literally titled: (null).
Two hosts requested /(null) in two different sub-directories (only; not top-level) and got 404s.

2.) Jim: Things match even sans extension. Via the nifty Firefox add-on "LiveHTTPHeaders [livehttpheaders.mozdev.org]":

A. GET /(null) HTTP/1.1
Content-Length: 69
Content-Type: text/html

B. GET /null HTTP/1.1
Content-Length: 446
Content-Type: text/html

GET /null.jpg HTTP/1.1
Content-Length: 708
Content-Type: image/jpeg

And thus "The Case of the Curious /(null) Calls" continues...
(CUE TWILIGHT ZONE THEME)

By way of a final update, or closure. (crosses fingers) No further requests for --

/(null)
/exampledir/(null)

- since Feb. 8. Curiously, four of the last five were rapid-fire from a single Trident/4.0 visitor:

[08/Feb/2009:10:22:51 -0800] "GET /(null) HTTP/1.1"
[08/Feb/2009:10:22:51 -0800] "GET /(null) HTTP/1.1"
[08/Feb/2009:10:22:52 -0800] "GET /exampledir/(null) HTTP/1.1"
[08/Feb/2009:10:23:05 -0800] "GET /exampledir/(null) HTTP/1.1"

That's reminiscent of a favicon checker...

Well, whatever. No signs of /(null) from Trident 4.0 since then.

Well, I've gotten 11 of them this month in my 404 log. The latest is two of them yesterday. All of them have Trident/4.0 in the UA, except for one. That one, requested /null instead of /(null) so I think that was an actual request.

A lot of these requests for /(null) get a second (or third) 404 within a few seconds. That looks to me like an actual person doing a reload from the 404 page.

P.S. Eight (null) requests thus far for March:

Six asked for /(null) and two for /exampledir/(null)
All had Trident/4.0 in their UA strings.
All were one-hit wonders.
All were from named hosts (not plain IP addresses); six were US-based.
(Hmm. The two non-US hosts made the /exampledir/(null) requests.)

FWIW: Something change somewhere (...vis-a-vis Trident)? Just 3/4ths of the way through 03-19, Pacific, my (null) request rate equals all of March to date.

----------
GET /(null)
----------

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 1.1.4322; InfoPath.2; .NET CLR 2.0.50727; OfficeLiveConnector.1.3; OfficeLivePatch.0.0)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.30618; .NET CLR 3.5.30729; OfficeLiveConnector.1.3; OfficeLivePatch.1.3)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2)

----------
GET /exampledirs/(null)
----------

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.21022; InfoPath.2; MS-RTC EA 2; MS-RTC LM 8; .NET CLR 3.5.30729; .NET CLR 3.0.30618)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; InfoPath.2; .NET CLR 3.5.30729; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; .NET CLR 3.0.30618; .NET CLR 1.1.4322)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB5; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)

----------
Signed,
She Who Appears To Be Talking To Herself;)

So, are you successfully blocking or redirecting these requests now?

If so, what code did you end up with?

Thanks,
Jim

Thanks for reading/asking, Jim! Turned out that all URIs containing null were one-hit wonders so now I basically 403 the request via a piped array. For example:

RewriteCond %{REQUEST_URI} (mailz¦makemoney¦null¦root¦roundcube) [NC]
(etc.)

Used to be, the /(null) hits were just 1 per host. Then two. Then a few same-host multiples. Now, most are multiples. First seen in Jan., 2009. Halfway through March, the total count was 8. Halfway through May? 68. Not earthshaking but still mysterious, still multiplying.

Trident/4.0 continues to be the only common denominator. Hosts are worldwide (always hosts, not IPs), with a superhuman hit rate, and only hit /(null):

cpc2-[etc].brig.cable.ntl.com
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

05/10 13:19:01 /subdirA/subdirB/(null)
05/10 13:19:05 /subdirC/(null)
05/10 13:19:06 /subdirA/(null)

c-[etc].hsd1.ca.comcast.net
Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

05/14 21:42:12 /(null)
05/14 21:42:12 /(null)
05/14 21:42:13 /(null)
05/14 21:42:13 /subdir/(null)
05/14 21:42:15 /subdir/(null)

-----
P.S.
I still 403 'em. Makes no difference. No one's ever home at the other end. Bookmarks? Spider bites?

P.P.S.
More /(null)-hitting UAs with a broad range of components and (relatively) limited commonalities:

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; InfoPath.2; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; MSN OptimizedIE8;ENGB)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; WOW64; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 1.1.4322; OfficeLiveConnector.1.3; OfficeLivePatch.1.3; .NET CLR 3.5.30729; .NET CLR 3.0.30618)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; eMusic DLM/4; .NET CLR 3.5.30729; .NET CLR 3.0.30618)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; [bsalsa.com)...] ; .NET CLR 1.1.4322; .NET CLR 2.0.50727; OfficeLiveConnector.1.3; OfficeLivePatch.0.0; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; Creative AutoUpdate v1.10.10)

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB6; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; MSN Optimized;US)

I've seen these "(null)" requests from time-to-time. But, not often enough to get onto my List-of-Things-I-Absolutely-Need-To-Address.

Looking back 9 days across 4 domains on the same web host:

69.137.232.xx - - [12/May/2009:19:43:09 -0400] "GET /W3DHJ/(null) HTTP/1.1" 404 1968 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)"
69.137.232.xx - - [12/May/2009:19:43:17 -0400] "GET /W3DHJ/(null) HTTP/1.1" 404 1968 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)"

$ rdns 69.137.232.xx
c-69-137-232-xx.hsd1.md.comcast.net

(IP redacted)

Jonesy

Update: 97 hits thus far this month to busiest site. More multiple, simultaneous /(null) hits from hosts; hit rates resemble a bot-like process, e.g., bookmark- or favicon-checker, etc. All hits 403'd. Along the way I 403'd one Comcast null'er for a different reason and when they e-mailed me, I asked them about the still apparently Trident-related hits. They had no clue. Neither do I.

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

06/19 23:35:26 /(null)
06/19 23:35:26 /(null)
06/19 23:35:27 /subdirA/(null)
06/19 23:35:28 /(null)
06/19 23:35:29 /subdirA/(null)
06/19 23:35:29 /(null)
06/19 23:35:29 /(null)

First I've seen of these.

99.248.189.zzz - - [01/Jul/2009:22:06:58 +0100] "GET /(null) HTTP/1.1" 404 1264 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; Avant Browser; Avant Browser; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 1.1.4322; .NET CLR 3.5.30729; .NET CLR 3.0.30618)"

99.248.189.zzz - - [01/Jul/2009:22:07:00 +0100] "GET /MyFolder/(null) HTTP/1.1" 404 1264 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; Avant Browser; Avant Browser; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 1.1.4322; .NET CLR 3.5.30729; .NET CLR 3.0.30618)"