<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <title>Plop in the Ocean</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/" />
    <link rel="self" type="application/atom+xml" href="http://pgregg.com/blog/atom.xml" />
    <id>tag:pgregg.com,2009-02-01:/blog//2</id>
    <updated>2011-08-25T23:02:50Z</updated>
    
    <generator uri="http://www.sixapart.com/movabletype/">Movable Type 4.34-en</generator>

<entry>
    <title>Amusing, True Near Death Experience.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2011/08/amusing-true-near-death-experience.html" />
    <id>tag:pgregg.com,2011:/blog//2.347</id>

    <published>2011-08-25T22:49:59Z</published>
    <updated>2011-08-25T23:02:50Z</updated>

    <summary>Those who know me will know that there have been two important events occurring in my life this week. 1) Tiling my kitchen, and 2) Trying to buy a HP TouchPad.As part of tiling the kitchen, significant other wanted to replace all the standard white wall sockets with Stainless Steel ones to match the rest of the kitchen colour scheme. I was safely over half way through this task, having disconnected the socket mains supply, when a van pulled up bearing HP TouchPad goodness. Thus the next 30 minutes of my life was accounted for.Upon returning to the kitchen tasks...</summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[Those who know me will know that there have been two important events occurring in my life this week. 1) Tiling my kitchen, and 2) Trying to buy a HP TouchPad.<br /><br />As part of tiling the kitchen, significant other wanted to replace all the standard white wall sockets with Stainless Steel ones to match the rest of the kitchen colour scheme. I was safely over half way through this task, having disconnected the socket mains supply, when a van pulled up bearing HP TouchPad goodness. Thus the next 30 minutes of my life was accounted for.<br /><br />Upon returning to the kitchen tasks I came to the Cooker 40A switch. I disconnected the ring main from the switch and proceeded to move the wires so that I kept the supply and load pairs apart when BUZZZZZZ and 240 Volts of high energy goodness shot through my hand and up my arm.<br /><br />Yes folks, TouchPad caused me to forget to flip the cooker ring circuit in the fuse box.<br /><br />Cue me jumping back, dropping tools and collapsing to the floor in a bout of Arrrrgh.<br /><br />Wife said "Stop it, you'll scare the kids.".<br /><br />Kids run in asking what happened and wife explains that Daddy electrocuted himself.<br /><br />Six year old Lauren asked "Could you see his bones?".<br /><br />
     ]]>
        
    </content>
</entry>

<entry>
    <title>What 16TB raw space looks like at home</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2011/04/what-16tb-raw-space-looks-like.html" />
    <id>tag:pgregg.com,2011:/blog//2.345</id>

    <published>2011-04-11T16:40:07Z</published>
    <updated>2011-04-11T18:44:39Z</updated>

    <summary>I&apos;ve been looking for some home backup solutions over the past couple of months. This has led me down both the do-it-yourself route and buying a ready-made solution.One of my requirements was that I wanted the solution to be more than just storage - otherwise I would have purchased a straight NAS box from the likes of Qnap, Netgear or if feeling rich Drobo. Most of these dedicated NAS boxes can be &quot;rooted&quot; to allow ssh access , however their CPUs are generally underpowered for general purpose use.Other requirements were that I wanted a reasonably small form factor and to...</summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="Systems" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="f4eg" label="F4EG" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="hpmicroserver" label="HP Microserver" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="linux" label="Linux" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="nas" label="NAS" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="networkattachedstorage" label="Network-attached storage" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="raid" label="RAID" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="sandiskcruzer" label="Sandisk Cruzer" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="serialata" label="Serial ATA" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="tb" label="TB" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[I've been looking for some home backup solutions over the past couple of months. This has led me down both the do-it-yourself route and buying a ready-made solution.<br /><br />One of my requirements was that I wanted the solution to be more than just storage - otherwise I would have purchased a straight NAS box from the likes of Qnap, Netgear or if feeling rich Drobo. Most of these dedicated NAS boxes can be "rooted" to allow ssh access , however their CPUs are generally underpowered for general purpose use.<br /><br />Other requirements were that I wanted a reasonably small form factor and to be able to use at least 4 SATA hard drives, preferably with hot swap ability. Hardware raid was not a requirement because I intended on using a Linux distribution with mdadm software raid.<br /><br />In the end, I ended up building two boxes.<br />The first, a home build, based on the <a href="http://linitx.com/viewproduct.php?prodid=12789&amp;sr=ts">CFI A7879 chassis</a> <a href="http://pgregg.com/blog/assets_c/2011/04/CFI_A7879_1-15.html" onclick="window.open('http://pgregg.com/blog/assets_c/2011/04/CFI_A7879_1-15.html','popup','width=550,height=380,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false"><img src="http://pgregg.com/blog/assets_c/2011/04/CFI_A7879_1-thumb-250x172-15.jpg" alt="CFI_A7879_1.jpg" class="mt-image-right" style="float: right; margin: 0pt 0pt 20px 20px;" height="172" width="250" /></a>with a <a href="http://www.mini-itx.com/store/?c=71#ga-d525tud">Gigabyte GA-D525TUD Dual Core Atom Mini-ITX Board</a>. <br /><a href="http://pgregg.com/blog/assets_c/2011/04/GA-D525TUD-18.html" onclick="window.open('http://pgregg.com/blog/assets_c/2011/04/GA-D525TUD-18.html','popup','width=200,height=150,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false"><img src="http://pgregg.com/blog/assets_c/2011/04/GA-D525TUD-thumb-200x150-18.jpg" alt="GA-D525TUD.jpg" class="mt-image-none" style="" height="150" width="200" /></a><br /><br /><div>The second was a off-the-shelf <a href="http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-4237916-4237918-4237917-4248009.html">HP ProLiant Microserver</a> which, to be brutally honest, was because HP were offering <a href="http://h41112.www4.hp.com/promo/proliantmicroserver/pdf/PA0023-%C2%A350_off_Microserver_Offer_Apr_2011.pdf">£100 cashback</a> deal on it. This made the server much cheaper than you could possibly build yourself from components.<br /><a href="http://pgregg.com/blog/assets_c/2011/04/HP_Microserver-21.html" onclick="window.open('http://pgregg.com/blog/assets_c/2011/04/HP_Microserver-21.html','popup','width=500,height=414,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false"><img src="http://pgregg.com/blog/assets_c/2011/04/HP_Microserver-thumb-250x207-21.jpg" alt="HP_Microserver.jpg" class="mt-image-right" style="float: right; margin: 0pt 0pt 20px 20px;" height="207" width="250" /></a><br /></div><div>I added 4GB ram to each box (total 5GB in the HP box because it comes with 1GB).<br /><br />The CFI boot drive is a 8GB (30MB/sec) CompactFlash card mounted as an 
IDE drive. The HP boot drive is a 16GB Sandisk Cruzer USB stick.<br />
<br />Finally added 4 x 2TB Samsung F4EG HD204UI drives to each box.<br /><br />The CFI box has 8TB in RAID5 providing 5.4TB usable. The HP has 8TB in RAID6 providing 3.6TB usable space.<br /><br />If there is more interest, I'll write up the build process is more detail with pictures.<br /><br />For now - here are some shots of my utility shelf.<br /><br /><a href="http://pgregg.com/blog/assets_c/2011/04/IMG_20110411_173250-24.html" onclick="window.open('http://pgregg.com/blog/assets_c/2011/04/IMG_20110411_173250-24.html','popup','width=1920,height=2560,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false"><img src="http://pgregg.com/blog/assets_c/2011/04/IMG_20110411_173250-thumb-240x320-24.jpg" alt="IMG_20110411_173250.jpg" class="mt-image-none" style="" height="320" width="240" /></a><a href="http://pgregg.com/blog/assets_c/2011/04/IMG_20110411_173740-27.html" onclick="window.open('http://pgregg.com/blog/assets_c/2011/04/IMG_20110411_173740-27.html','popup','width=1920,height=2560,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false"><img src="http://pgregg.com/blog/assets_c/2011/04/IMG_20110411_173740-thumb-250x333-27.jpg" alt="IMG_20110411_173740.jpg" class="mt-image-none" style="" height="333" width="250" /></a><br /><br /><br /></div>

]]>
        
    </content>
</entry>

<entry>
    <title>Experts Exchange, Google, AllFAQ.org and misappropriation of copyright.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2011/01/experts-exchange-google-allfaqorg-dmca.html" />
    <id>tag:pgregg.com,2011:/blog//2.344</id>

    <published>2011-01-19T21:50:45Z</published>
    <updated>2011-01-20T14:50:00Z</updated>

    <summary>Opinion PieceI was googling (as a verb) and came across a rather peculiar message at the bottom of Google&apos;s search results:In response to a complaint we received under the US Digital Millennium Copyright Act, we have removed 1 result(s) from this page. If you wish, you may read the DMCA complaint that caused the removal(s) at ChillingEffects.org.Interesting - never saw that before!Following the link to Chilling Effects shows a copy of the complaint which has some interesting text in it.Experts-Exchange makes a detailed itemisation of their registered Copyrights, none of which I find objectionable, however, the complaint then goes on...</summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="General" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Software" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="allfaqorg" label="allfaq.org" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="copyright" label="copyright" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="dmca" label="DMCA" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="expertsexchange" label="experts-exchange" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="google" label="google" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[Opinion Piece<br /><br />I was googling (as a verb) and came across a rather peculiar message at the bottom of Google's search results:<br /><br /><blockquote><span style="visibility: visible;" id="xjs"><i>In response to a complaint we received under the <a href="http://www.google.co.uk/intl/en/dmca.html">US Digital Millennium Copyright Act</a>, we have removed 1 result(s) from this page. If you wish, you may <a href="http://www.chillingeffects.org/notice.cgi?sID=30399">read the DMCA complaint</a> that caused the removal(s) at ChillingEffects.org.</i></span><br /><span style="visibility: visible;" id="xjs"></span></blockquote><span style="visibility: visible;" id="xjs"><i><br /></i></span>Interesting - never saw that before!<br /><br />Following the link to <a href="http://www.chillingeffects.org/notice.cgi?sID=30399">Chilling Effects</a> shows a copy of the complaint which has some interesting text in it.<br /><br />Experts-Exchange makes a detailed itemisation of their registered Copyrights, none of which I find objectionable, however, the complaint then goes on to list several issues against the Defendant, the first and most egregious of which is:<br /><br /><blockquote><i>a direct "copy and paste job" lifting the content of Plaintiff's question and answer forums and inserting them onto AllFAQ's website. AllFAQ's question and "Solutions" are verbatim to Experts-Exchange's questions and "Accepted Solutions;" </i><br /></blockquote><br />From this <a href="http://experts-exchange.com/">Experts Exchange</a> is accusing <a href="http://allfaq.org/">allfaq.org</a> of Copyright infringement against Experts Exchange owned Copyright.<br /><br />At first glance, this might seem fully justified - but look at what they are claiming copyright on.&nbsp; Experts Exchange are assuming copyright ownership of content that you, and I, and all their users create by asking and answering questions on their web site.<br /><br />I looked at Experts Exchange's <a href="http://www.experts-exchange.com/termsOfUse.jsp">Terms of Use</a> and could not find any agreement that users were assigning their rights and copyrights to Experts Exchange. The relevant paragraph is:<br /><br /><blockquote>"5. Content License<dl><dd>
         <p>
         EXPERTS EXCHANGE enables Members to post problems or questions, 
         proposed solutions or answers, information, comments and other content 
         ("Your Content") to its Site.  When you post Your Content to the Site, 
         you understand and agree that Your Content can be viewed and used by 
         other Members who visit the Site with or without attribution.
         </p>
         
         <p>
         You represent and warrant that you own or otherwise control all of the 
         rights to Your Content and that use of Your Content by EXPERTS 
         EXCHANGE and its affiliates will not infringe upon or violate the 
         rights of any third party. Before you use EXPERTS EXCHANGE Services to 
         post any information or content that is protected by intellectual 
         property laws, you shall have acquired the legal right to do so from 
         the owner or authorized licensee of such information or content.
         </p>
         
         <p>
         By registering with EXPERTS EXCHANGE and posting Your Content on the 
         Site, you hereby: (i) grant EXPERTS EXCHANGE a non-exclusive,
         perpetual, irrevocable, unrestricted, transferable, fully
         sub-licensable, worldwide, royalty-free license to use, distribute,
         display, reproduce, perform, modify, adapt, publish, translate and
         create derivative works from Your Content in any form, media or
         technology, whether now-known or hereafter developed; (ii) grant
         EXPERTS EXCHANGE and its affiliates and sub-licensees the right to use
         the Member Name that you submit with Your Content for purposes of
         attribution; (iii) authorize EXPERTS EXCHANGE to assert and prosecute
         claims against any third-party making any unauthorized use of Your
         Content, including any use that violates this User Agreement
         ("Third-Party Claims"); and (iv) appoint EXPERTS EXCHANGE as your
         attorney-in-fact for the purpose of asserting and prosecuting
         Third-Party Claims. If you do not wish to have Your Content attributed
         to you, then you must notify EXPERTS EXCHANGE at
            <a href="mailto:customer_service@experts-exchange.com">
            customer_service@experts-exchange.com</a>.
         </p>
      </dd></dl></blockquote><br />Experts Exchange acknowledges that the copyright belongs to the author as "Your Content" and that by posting you are granting them extensive <b>licenses</b> to use that content. You are not assigning your copyright to Experts Exchange.<br /><br />Now I am glad that their ToU does not attempt to wrest copyright ownership from its rightful owner, that is right and proper.<br /><br />allfaq.org is demonstrably guilty of screen-scraping the Experts 
Exchange web site and I do not condone those actions at all. However, looking at what 
they copied - it was the Title, Question and Accepted Solution text - 
the copyright of 100% of that is with the original authors, and not 
Experts Exchange.<br />
<br />Thus, in my opinion, this complaint against allfaq.org is without merit and should be dismissed.<br /><br />It would also appear that Experts Exchange has also abused the provisions of the DMCA in forcing Google to remove the content. Google should restore the links.<br /><br />And finally, Experts Exchange should implement some technical measures to prevent automated scraping. Find better ways to improve your search ranking, and if your competition beats you don't <a href="http://www.experts-exchange.com/Web_Development/Internet_Marketing/Search_Engine_Optimization_SEO/Q_26271117.html">ask your own members how to do better SEO</a>; be told by them that you have no Copyright Claims on the content; and then proceed to file DMCA take down notices when you know you have no (copy)right.<br /><br /><br /><br /><br />]]>
        
    </content>
</entry>

<entry>
    <title>Stage 2: http:BL with Apache2 mod_perl</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2010/12/stage-2-httpbl-with-apache2-mod-perl.html" />
    <id>tag:pgregg.com,2010:/blog//2.343</id>

    <published>2010-12-02T11:19:45Z</published>
    <updated>2010-12-02T12:31:11Z</updated>

    <summary><![CDATA[After my earlier post Referrer and Comment spammers are a PITA I came up with two mod_perl plugins to Apache and an "apache level" firewall.The reason for the apache-level firewall is two-fold.&nbsp; There is no direct way for the Apache user to manipulate an iptables chain (as it doesn't run as root), and second; I was not happy with suid root access or other forms of message passing to a daemon which would manipulate the firewall for me.Architecture is thus, in httpd.conf place the following two lines:PerlPreConnectionHandler PGREGG::httpBLBlockPerlLogHandler PGREGG::httpBLLogThe first tells apache to run the handler in my httpBLBlock.pm module...]]></summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="PHP" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Perl" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="apache" label="apache" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="firewall" label="firewall" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="mod_perl" label="mod_perl" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="perl" label="perl" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="php" label="php" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[After my earlier post <a href="http://pgregg.com/blog/2010/11/referrer-and-comment-spammers-are-a-pita.html">Referrer and Comment spammers are a PITA</a> I came up with two mod_perl plugins to Apache and an "apache level" firewall.<br /><br />The reason for the apache-level firewall is two-fold.&nbsp; There is no direct way for the Apache user to manipulate an iptables chain (as it doesn't run as root), and second; I was not happy with suid root access or other forms of message passing to a daemon which would manipulate the firewall for me.<br /><br />Architecture is thus, in httpd.conf place the following two lines:<br /><blockquote>PerlPreConnectionHandler PGREGG::httpBLBlock<br />PerlLogHandler PGREGG::httpBLLog<br /></blockquote>The first tells apache to run the handler in my httpBLBlock.pm module when a connection is received (before the request has been sent by the client).&nbsp; In this handler, I am simply looking for a filename matching that IP in a directory that is writable by the apache user.&nbsp; The contents of the file are a SCORE:httpBL_answer:[LIST].&nbsp; Based on this, the module checks the mtime of the filename is in the last SCORE days, then the firewall is in effect. If so, we simply tell apache to drop the connection.&nbsp; If the file has expired, we delete the file.<br /><br />The second line is more interesting, and what creates the firewall filenames. In order to not impede the general speed of request handling, processing is performed in the Logging section of the Apache process. Our module is called by apache after the response has been sent, but before the access_log entry has been written.&nbsp; In our module we perform the <a href="http://www.projecthoneypot.org/httpbl_api.php">http:BL</a> API call and compute the above SCORE based upon the Threat* level and Age* of the API response. (* both Threat and Age are octets in the DNS lookup).&nbsp; We merely discount the Threat down to zero based on the Age (0-255) where an entry 255 days old reduces the SCORE to zero.<br />If the SCORE is larger than our trigger level (3) then we create the firewall filename, log the entry in our own httpbl.log and return Apache2::Const::FORBIDDEN.&nbsp; This causes Apache to not log the entry in the normal access_log.&nbsp; Otherwise, if all is ok, we return Apache2::Const::OK and Apache logs the hit as normal.<br /><br />I have a bit of code tidy up, restructure the config/firewall directory and pull some common code out to a shared module before I can release to the world.<br /><br />An interesting side effect to publishing the last story out through <a href="http://www.planet-php.net/">Planet PHP</a> and other news sources <i>along with the Project Honey Pot image</i> is that when browsers viewed those sources, they all asked for the image off my server. In several cases, these were known spammer, Comment spammer, and other abusers. My server then created the firewall entry blocking them before they were able to follow the links back to my server.<br />&nbsp;<br />I have been reading up more on <a href="http://perl.apache.org/docs/2.0/api/Apache2/Filter.html">Apache Bucket Brigades</a> in an attempt to allow the firewall filter to be placed immediately after the request has been received and allow a custom response to the browser. This may help an otherwise unsuspecting user if their machine had been trojaned. I don't mind admitting I'm thoroughly confused right now :)<br /><br />]]>
        
    </content>
</entry>

<entry>
    <title>Referrer and Comment spammers are a PITA.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2010/11/referrer-and-comment-spammers-are-a-pita.html" />
    <id>tag:pgregg.com,2010:/blog//2.342</id>

    <published>2010-11-28T23:18:54Z</published>
    <updated>2010-11-28T23:45:53Z</updated>

    <summary><![CDATA[This shouldn't be news to anyone - but Referrer and Comment spammers are a real pain in the a*se.&nbsp; Polluting my web logs and making any meaningful log analysis problematic.So, I now have an itch to scratch and I'm going to do something about it. I would encourage you, the reader, to do something about it too.Firstly, get yourself over to Project Honey Pot and read up on the project.&nbsp; If you can, set up a Honey Pot or two yourself. Also be sure to read about the http:BL - this works along similar lines to the DNS blacklists used...]]></summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="PHP" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Perl" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="httpbl" label="http:BL" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="spam" label="spam" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[This shouldn't be news to anyone - but Referrer and Comment spammers are a real pain in the a*se.&nbsp; Polluting my web logs and making any meaningful log analysis problematic.<br /><br />So, I now have an itch to scratch and I'm going to do something about it. I would encourage you, the reader, to do something about it too.<br /><br />Firstly, get yourself over to <a href="http://www.projecthoneypot.org/?rf=73568">Project Honey Pot</a> and read up on the project.&nbsp; If you can, set up a Honey Pot or two yourself. Also be sure to read about the <a href="http://www.projecthoneypot.org/faq.php#g">http:BL</a> - this works along similar lines to the DNS blacklists used for Email spammers.<br /><br />Next, I'm going to write a general Apache mod_perl module which will provide integration (lookup) to the http:BL and allow the user to "action"* the abusers.&nbsp; Minimally, it will prevent the normal apache log files from being polluted by diverting the log entries to a httpbl logfile.<br /><br />* "action" - To provide flexibility, I'm thinking of running an external script with the IP of the abuser.&nbsp; The script can then perform any action you wish. The one I'm going for is an iptables firewall block.<br /><br />Comments and suggestions welcome.<br /><br />Project Honey Pot has implementations for several languages, including <a href="http://www.php.net/">PHP</a> and <a href="http://www.perl.org/">Perl</a> (the languages that mean most to me).&nbsp; There may be an implementation for your <a href="http://www.projecthoneypot.org/httpbl_implementations.php">Web application </a> so you might not be interested in what I'm doing at all :)<br /><br /><span class="mt-enclosure mt-enclosure-image" style="display: inline;"><img alt="projhoneypot_banner.jpg" src="http://pgregg.com/blog/images/projhoneypot_banner.jpg" class="mt-image-none" style="" height="60" width="468" /></span><br /><br /><br /><div><br /></div>]]>
        
    </content>
</entry>

<entry>
    <title>Vodafone UK + HTC Desire + Android 2.2 FroYo = Fail.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2010/08/vodafone-uk-htc-desire-android-22-froyo-fail.html" />
    <id>tag:pgregg.com,2010:/blog//2.341</id>

    <published>2010-08-04T17:59:03Z</published>
    <updated>2010-08-06T12:40:32Z</updated>

    <summary><![CDATA[Well, it looks like Vodafone UK royally messed up the timing of the HTC Desire OTA Update yesterday.&nbsp; While most owners were eagerly expecting the announced Android 2.2 (FroYo) update that HTC have been pushing out, Vodafone decided to push out a 2.1-update1 which only provides Vodafone branding, apps, a few bugs, and even a couple of "adult" related bookmarks to everyone - and leaving them on Android 2.1 (Eclair).Needless to say - Users are not happy at all. *Vodafone appears to have moved the thread here. [2010/08/06] Moved again to here (is Vodafone trying to hide the complaints?).In work,...]]></summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="General" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="android" label="android" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="froyo" label="froyo" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="htcdesire" label="htc desire" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="vodafone" label="vodafone" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[Well, it looks like Vodafone UK royally messed up the timing of the HTC Desire OTA Update yesterday.&nbsp; While most owners were eagerly expecting the announced Android 2.2 (FroYo) update that HTC have been pushing out, Vodafone decided to push out a 2.1-update1 which only provides Vodafone branding, apps, a few bugs, and even a couple of "adult" related bookmarks to everyone - and leaving them on Android 2.1 (Eclair).<br /><br />Needless to say - Users are <strike><a href="http://forum.vodafone.co.uk/topic/70196-htc-desire-new-firmware-update-but-not-22/">not happy</a></strike> at all. *Vodafone appears to have moved the thread <strike><a href="http://forum.vodafone.co.uk/topic/70507-htc-desire-new-firmware-but-not-22/">here</a></strike>. [2010/08/06] Moved again to <a href="http://forum.vodafone.co.uk/topic/70551-htc-desire-new-firmware-but-not-22/">here</a> (is Vodafone trying to hide the complaints?).<br /><br />In work, we (coworkers and I) now have a total of 8 HTC Desires (out of 16 people) - even two iPhone users have converted! A few have been espousing the wonders of FroYo on their phone (some had rooted, and did it themselves, others had bought unbranded phones and got the stock HTC FroYo upgrade). I was jealous(ish) and wanted it myself.<br /><br />Given Vodafone's actions yesterday, it was likely that the FroYo update from Vodafone was minimum several weeks away, if not 3 months (as was intimated on the Vodafone forum - end of October) - it was also noted that the Vodafone FroYo update would include the Vodafone 360 branding and software.<br /><br />Thus, last night it was time to embark on the adventure of flashing my HTC Desire to a stock image direct from HTC. I reasoned that my goal should be to flash HTC's Android 2.1 (Eclair) image to the phone, and once there, the normal software update process should take me to 2.2.<br /><br />And so it began...<br /><br />I knew that I needed to create a gold card because the Vodafone image doesn't let you install non-Vodafone images. This I discovered as I tried to simply apply a stock 2.2 download and using boot recovery update.zip - all attempts met with a complaint of a "Fingerprint error".<br /><br />I ended up at this page:<br />&nbsp; <a href="http://forum.xda-developers.com/showthread.php?t=696189">[TUT]Complete upgrading guide(root, unroot, flashing ROM &amp; updates)</a><br /><br />and followed <a href="http://forum.xda-developers.com/showpost.php?p=6710085&amp;postcount=3">Post #3</a> which details a) How to make a Gold Card* . Ignore the rest of #3<br />and then <a href="http://forum.xda-developers.com/showpost.php?p=6710092&amp;postcount=4">Post #4</a> - the "unrooting" guide.<br /><br />You then need to find a download of the correct stock HTC Bravo WWE image ROM - I chose the second WWE ROM from this link (140MB download):<br />&nbsp; <a href="http://forum.xda-developers.com/showthread.php?t=695667">[ROM] Official HTC Desire RUU ROMS and OTA Update URLs</a><br />&nbsp; RUU_Bravo_HTC_WWE_1.21.405.2_Radio_32.36.00.28U_4.06.00.02_2_release_126984_signed.exe<br /><br />Proceed through the rest of Post #4 "How to Flash ROM" with the Goldcard inside your Desire, and plugged into your PC.<br /><br />The phone will take 5-10 minutes to complete upgrading (or downgrading) to HTC Stock 2.1 image. When it reboots you will have to go through all the original setup sequences you did when you first unboxed your phone.<br /><br />You can then perform a Software Update Check and you should find you have a Android 2.2 FroYo update (90Mb) waiting for you.&nbsp; Proceed and let it do its thing.<br /><br />Once done, welcome to FroYo.<br /><br />All future updates will come direct from HTC - not from Vodafone, and you won't ever have the Vodafone 360 branding rubbish foisted upon you.<br /><br /><br />* My GoldCard creation had a bit of a hiccup, in that it turns out that the 4GB Samsung card which came with my Desire does not work as a Goldcard despite formatting and following the instructions to the letter.&nbsp; Trying an old 1GB Sandisk I had resulted in a good goldcard.<br /><br />Aside from the goldcard hiccup - this all went surprisingly smoothly and painlessly.<br /><br />Good luck.<br /><br />Disclaimer: If you try any of the above - it is all your responsibility. I take no responsibility should you brick or damage your phone.<br /><br /><div><br /></div>]]>
        
    </content>
</entry>

<entry>
    <title>The O2 Joggler - A first hack.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2010/04/the-o2-joggler---a-first-hack.html" />
    <id>tag:pgregg.com,2010:/blog//2.340</id>

    <published>2010-04-14T22:06:47Z</published>
    <updated>2011-04-16T16:05:25Z</updated>

    <summary><![CDATA[If you were not aware - O2 last week reduced the price of the O2 Joggler from £149.99 to £99.99 to £49.99.&nbsp; Nothing remarkable in price reductions, however what is remarkable is what you actually get for your money.The O2 Joggler is a silent 7" touchscreen device with Intel Atom Z520 running at 1.3Ghz, 512Mb ram, 1GB internal flash storage with additional storage available via an external USB port. What makes the device really exciting is that it also has a 1GB ethernet (Realtec 8168) and Wifi.Looking at that - I know you're all thinking "linux box" - but conveniently,...]]></summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="Joggler" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Software" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="joggler" label="joggler" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="o2joggler" label="o2 joggler" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="rtorrent" label="rtorrent" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[<span class="mt-enclosure mt-enclosure-image" style="display: inline;"><a href="http://pgregg.com/blog/assets_c/2010/04/devicev2-11.html" onclick="window.open('http://pgregg.com/blog/assets_c/2010/04/devicev2-11.html','popup','width=308,height=149,scrollbars=no,resizable=no,toolbar=no,directories=no,location=no,menubar=no,status=no,left=0,top=0'); return false"><img src="http://pgregg.com/blog/assets_c/2010/04/devicev2-thumb-308x149-11.jpg" alt="devicev2.JPG" class="mt-image-right" style="float: right; margin: 0pt 0pt 20px 20px;" height="149" width="308" /></a></span><br />If you were not aware - O2 last week reduced the price of the O2 Joggler from <strike>£149.99 </strike>to <strike>£99.99</strike> to £49.99.&nbsp; Nothing remarkable in price reductions, however what is remarkable is what you actually get for your money.<br /><br />The O2 Joggler is a silent 7" touchscreen device with Intel Atom Z520 running at 1.3Ghz, 512Mb ram, 1GB internal flash storage with additional storage available via an external USB port. What makes the device really exciting is that it also has a 1GB ethernet (Realtec 8168) and Wifi.<br /><br />Looking at that - I know you're all thinking "linux box" - but conveniently, the default operating system on it is based on Ubuntu 8.04 and busybox, The frontend is a custom flash driven UI developed by OpenPeak (makers of the Jogger which is rebadged by O2).<br /><br />There are plenty of clever individuals over at <a href="http://hackthejoggler.freeforums.org/">http://hackthejoggler.freeforums.org/</a>and in the #mer channel on freenode who are working to allow the Joggler to run other Operating Systems such as Android, Ubuntu Netbook Edition, MID, even Windows and OSX.<br /><br />However, the standard OS supplied is also pretty powerful and allows significant customisation.&nbsp; You can <a href="http://jogglerwiki.info/index.php?title=Installing_Telnet">enable telnet</a> just by turning it on with a custom USB stick plugged in. To do some of the things I'll be describing here you will need to have started with that.<br /><br />Once you can telnet in, the world opens up and you can do lots of things that you would expect* to be able to do in a linux system. (<i>* except run a web browser.... at this time - we don't have web access 
with the stock OS due to the custom flash GUI interface.</i>)<br /><br />Lets install some useful utilities, how about perl, terminfo, irc clients, bit torrent, rsync, ssh/scp ?&nbsp;&nbsp; Yes, I know what you're thinking - overnight silent downloads - no need to leave PC on... nice...<br /><br />I've built several of these utilities with installation instructions and uploaded them over here:&nbsp; <br />&nbsp; <a href="http://pgregg.com/projects/joggler/">http://pgregg.com/projects/joggler/</a><br /><br />Feel free to download, examine, install etc.&nbsp; Read the README on the download page - it explains how to do it. Most of the packs include an install.sh script that will provide an installation safety net - it won't overwrite existing files or libraries.<br /><br />Usual disclaimers apply - you try these applications at your own risk. I accept no responsibility if you manage to brick, fry or otherwise trash your joggler.<br /><br />Comments, suggestions for other applications are always welcome.<br /><br /><b>Update</b>: Thanks to NP - seems I was a little too brutal with the library stripping to keep the download sizes small and I missed some required libraries (that I mistakenly thought were in the standard Joggler distro).&nbsp;&nbsp; I have rebuilt the following packages as they were missing some libs: rtorrent, rsync, sudo,ssh<br />The screen package has been rebuilt too to add one final tweak (to the installer script) so it can be used by non-root users - the only difference you need to do is run the command: chmod 666 /dev/ptmp /dev/tty<br /><br />]]>
        
    </content>
</entry>

<entry>
    <title>Is BT still collecting usage data?</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2010/03/is-bt-still-collecting-usage-data.html" />
    <id>tag:pgregg.com,2010:/blog//2.339</id>

    <published>2010-03-31T07:30:00Z</published>
    <updated>2010-03-31T13:28:17Z</updated>

    <summary><![CDATA[After the recent scandal over BT using Phorm (see here and here) is it still possible that BT is still covertly collecting data profiling its user's internet usage?I have, as part of my home BT Total Broadband package, a HUAWEI Mobile Connect 3G Modem.&nbsp; This is a USB dongle that provides internet service.&nbsp;&nbsp; I rarely use it, but because of the storms last night my internet connection has been very unstable, therefore I plugged it in today.When I started the "BT Connection Manager" software, it informed me that there was an update to the service and would I like to...]]></summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="General" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Software" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="bt" label="BT" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="btconnectionmanager" label="BT Connection Manager" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="privacy" label="privacy" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[After the recent scandal over <a href="http://www.bt.com/">BT</a> using <a href="http://en.wikipedia.org/wiki/Phorm">Phorm</a> (see <a href="http://www.theregister.co.uk/2010/02/25/bt_cps/">here</a> and <a href="http://www.theinquirer.net/inquirer/news/1051760/eu-slaps-uk-phorm-fiasco">here</a>) is it still possible that BT is still covertly collecting data profiling its user's internet usage?<br /><br />I have, as part of my home BT Total Broadband package, a <a href="http://www.productsandservices.bt.com/consumerProducts/displayCategory.do?categoryId=CON-MOBILE-R1">HUAWEI Mobile Connect 3G Modem</a>.&nbsp; This is a USB dongle that provides internet service.&nbsp;&nbsp; I rarely use it, but because of the storms last night my internet connection has been very unstable, therefore I plugged it in today.<br /><br />When I started the "BT Connection Manager" software, it informed me that there was an update to the service and would I like to download it? Yes, ok, and a couple of minutes later it began to install itself.&nbsp; Then popped up the obligatory <a href="http://en.wikipedia.org/wiki/Software_license_agreement">EULA</a> which contained the paragraphs:<br /><br /><blockquote>"<i>Installation of this Software will automatically apply a unique identifier to the Software, this identifying feature will only be used in accordance with the BT privacy policy which can be found at www.bt.com.</i><br /><br /><i>This software automatically collects session and client parameter Information for all connections made via this Software. This information is automatically transmitted to BT at the start of each session where Internet access is available. This session information will only be used in accordance with the BT privacy policy which can be found at www.bt.com."</i><br /><br /><span class="mt-enclosure mt-enclosure-file" style="display: inline;"><a href="http://pgregg.com/blog/images/BTConnectionManagerEULA.txt">BTConnectionManagerEULA.txt</a></span><br /></blockquote><div>I also note that the software package uses the Open Source packages libCurl, openssl and sqlite.&nbsp; Combined they enable the package to record "session and client parameter Information for all connections" in the database (sqlite) and to encrypt (openssl) and transmit the data via a web request (libcurl) back to BT.<br /><br />At this point I am unable to confirm if such a transmission is taking place as I declined the EULA, but would welcome feedback or comments from others that are able to investigate this more fully.<br /><br />Update: Even though I clicked "Cancel" to reject the EULA - it did update the application and actually started it up before I exited the installer.&nbsp; Guess I have this privacy invading feature now.<br /><br /><br /><br /></div><div><br /></div>]]>
        
    </content>
</entry>

<entry>
    <title>Amazon 1 - 0 Waterstones, aka this morning&apos;s rant.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2009/09/amazon-1---0-waterstones-aka-this-mornings-rant.html" />
    <id>tag:pgregg.com,2009:/blog//2.338</id>

    <published>2009-09-29T10:02:59Z</published>
    <updated>2009-09-29T10:24:10Z</updated>

    <summary><![CDATA[SWMBO asked me to order a book for her this morning, so being the tightwad that I am, I go hunting for the ISBN and the cheapest place to buy it.Amazon has it for £17.24 and Waterstone's for £18.49 (but if I order through quidco, then I receive another 8% cashback, making Waterstone's cheaper).Waterstone's need me to create an account, fair enough and state "Also, please be aware that passwords are case sensitive, and must be a minimum of seven characters long and at least one character must be numeric."&nbsp; Again, all fairly standard.So each time I try to enter...]]></summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="General" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="amazon" label="amazon" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="rant" label="rant" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="waterstones" label="waterstones" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[SWMBO asked me to order a book for her this morning, so being the tightwad that I am, I go hunting for the ISBN and the cheapest place to buy it.<br />Amazon has it for £17.24 and Waterstone's for £18.49 (but if I order through <a href="http://www.quidco.com/">quidco</a>, then I receive another 8% cashback, making Waterstone's cheaper).<br /><br />Waterstone's need me to create an account, fair enough and state "Also, please be aware that passwords are case sensitive, and must be a
minimum of seven characters long and at least one character must be
numeric."&nbsp; Again, all fairly standard.<br /><br />So each time I try to enter a password which meets this criteria I'm hit with a javascript popup that claims: "Your new password must be at least 7 characters long and contain at least one digit."<br /><br />"But it is!", I yell in frustration.<br /><br />Using the Firefox "Web Developer" plugin I unhide the password boxes and see that my password doesn't have digits... wtf!&nbsp;&nbsp; I enter the digits again and find there is another hidden rule:<br /><pre id="line696">&lt;<span class="start-tag">input</span><span class="attribute-name"> name</span>=<span class="attribute-value">"newPassword" </span><span class="attribute-name">type</span>=<span class="attribute-value">"password"<br /></span><span class="attribute-name">id</span>=<span class="attribute-value">"fPassword" </span><span class="attribute-name">size</span>=<span class="attribute-value">"40" </span><span class="attribute-name">maxlength</span>=<span class="attribute-value">"16" </span><span class="attribute-name">value</span>=<span class="attribute-value">"" </span><span class="error"><span class="attribute-name">/</span></span>&gt;<br /></pre>Yes, an upper limit of 16 characters on the password.<br /><br />Right ok, make a password of 16 chars or under and move on...<br /><br />Add name, postcode to the next form and it finds my address ok - click continue to (I presume) proceed to the CC entry form and voila!<br /><br />Well not quite.<br /><br />"There has been a problem processing this request
<br />Please use the refresh button on your browser to try again.<br /><br />Thank you."<br /><br />A few refreshes later, it is apparent that Waterstone's has no intention of working, sod it, for 20p more I can have less hassle at Amazon.<br /><br /><br /><br /><br /><br />]]>
        
    </content>
</entry>

<entry>
    <title>Getty Images / BBC Math Fail.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2009/06/getty-images-bbc-math-fail.html" />
    <id>tag:pgregg.com,2009:/blog//2.336</id>

    <published>2009-06-03T19:50:50Z</published>
    <updated>2009-06-03T20:13:07Z</updated>

    <summary>From the BBC Magazine today is an article 7 questions on GCSE maths.Can you spot the fail? Yes, epic....</summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="General" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="bbc" label="bbc" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="fail" label="fail" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="funny" label="funny" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="math" label="math" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[From the <a href="http://news.bbc.co.uk/1/hi/magazine/">BBC Magazine</a> today is an article <a href="http://news.bbc.co.uk/1/hi/magazine/8081043.stm">7 questions on GCSE maths</a>.<br /><br />Can you spot the fail?<br /><br /><span class="mt-enclosure mt-enclosure-image" style="display: inline;"><img alt="bbc_math_fail1.png" src="http://pgregg.com/blog/images/bbc_math_fail1.png" class="mt-image-none" style="" width="431" height="419" /></span><br /><img src="file:///C:/DOCUME%7E1/pgregg/LOCALS%7E1/Temp/moz-screenshot.jpg" alt="" /><img src="file:///C:/DOCUME%7E1/pgregg/LOCALS%7E1/Temp/moz-screenshot-1.jpg" alt="" /><br /><br /> <div>Yes, epic.<br /><br /><span class="mt-enclosure mt-enclosure-image" style="display: inline;"><img alt="bbc_math_fail2.png" src="http://pgregg.com/blog/images/bbc_math_fail2.png" class="mt-image-none" style="" width="216" height="182" /></span><br /></div><div><br /></div>]]>
        
    </content>
</entry>

<entry>
    <title>Register your broadband not-spot now</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2009/05/register-your-broadband-not-spot-now.html" />
    <id>tag:pgregg.com,2009:/blog//2.335</id>

    <published>2009-05-28T08:00:00Z</published>
    <updated>2009-05-28T13:34:47Z</updated>

    <summary>thinkbroadband, formerly ADSLGuide.org.uk, have setup a community site where UK sufferers of the lack of universal broadband can register their inability to obtain broadband (a not-spot) or where you are unable to obtain up to 2Mbps service (slow-spot).The Northern Ireland map is here - if you currently suffer from a lack of, or poor broadband coverage (including satellite) please register your details on the site - by adding your postcode it will appear on the map and we can begin to get a true picture of the poor state of broadband in Northern Ireland.Thanks....</summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="General" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="broadband" label="broadband" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="detini" label="detini" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="ni" label="ni" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="northernireland" label="northern ireland" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[<a href="http://www.thinkbroadband.com/">thinkbroadband</a>, formerly ADSLGuide.org.uk, have setup a community site where UK sufferers of the lack of universal broadband can register their inability to obtain broadband (a not-spot) or where you are unable to obtain up to 2Mbps service (slow-spot).<br /><br />The <a href="http://www.broadband-notspot.org.uk/coverage-map.html#9,54.61820741719788,-6.4654541015625,">Northern Ireland</a> map is here - if you currently suffer from a lack of, or poor broadband coverage (including satellite) please register your details on the site - by adding your postcode it will appear on the map and we can begin to get a true picture of the poor state of broadband in Northern Ireland.<br /><br />Thanks.<br /><div><br /></div><div><br /></div>]]>
        
    </content>
</entry>

<entry>
    <title>Borland, Farewell my sweet.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2009/05/borland-farewell-my-sweet.html" />
    <id>tag:pgregg.com,2009:/blog//2.334</id>

    <published>2009-05-06T14:31:36Z</published>
    <updated>2009-05-06T17:34:46Z</updated>

    <summary>Today, May 6 2009, marks the day that Borland, that once great master of all software development has finally recognised there was no other recourse but to up and sell itself off in order to survive.Back when I was a teenager, in the early 80s and personal computing was coming to the fore - I, and many others, aspired to work for that great company Borland. It was the pinnacle of language development and development tools and we wanted to work there. However, based in Ireland it was never to be.Also, once upon a time I happened to be working...</summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="General" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="PHP" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Software" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="acquisitions" label="acquisitions" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="borland" label="Borland" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="delphi" label="delphi" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="jbuilder" label="jbuilder" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="ma" label="M&amp;A" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="software" label="software" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[Today, May 6 2009, marks the day that Borland, that once great master of all software development has finally recognised there was no other recourse but to up and sell itself off in order to survive.<br /><br />Back when I was a teenager, in the early 80s and personal computing was coming to the fore - I, and many others, aspired to work for that great company Borland. It was the pinnacle of language development and development tools and we wanted to work there. However, based in Ireland it was never to be.<br /><br />Also, once upon a time I happened to be working for a very promising young company with a fantastic product line called Segue Software, based in Boston, MA.&nbsp; Segue also had its troubles but a new CEO saw its fortunes turn and it was climbing to success. This success was noticed by the aforementioned Borland as it tried to re-invent itself as an Application Lifecycle Management (ALM) company.&nbsp; The same day it was announced that Borland was acquiring Segue, it also announced it was selling its developer tools division (that's Delphi, JBuilder, and later Delphi for PHP, 3rd Rail line of products).<br /><br />This was such a bitter-sweet time for many.&nbsp; I was overjoyed that I was going to work for Borland (childhood dream) - it didn't matter I wasn't going to work with the developer tools, working for "Borland" would just be cool.&nbsp; Sadness also because our little 200-man company was being consumed by a 1200-man behemoth (relatively) and no-matter which way you looked at it, people were going to lose their jobs.&nbsp; Pretty much the entire US East Coast staff (Segue Head Office) lost their jobs and the office was closed.&nbsp; Product development labs and Technical Support survived, simply by virtue that it was the product and product skills that were purchased, not the G&amp;A functions - they could go.<br /><br />I 'lived the dream' for the next 2.5 years in the IT department. Despite being remote, I loved working with the rest of the Borland teams as I was intimately involved in the merging/migration of Segue's systems into Borland's. I also had the pleasure of working with several departments to architect and deploy several new platforms (such as product downloads and licensing via Intraware, and the companies Salesforce.com, SFDC, deployment).&nbsp; I'll treasure the time I spent at Borland.<br /><br />Of course there were several WTF moments. Most significantly, for me, was the company "hanging its hat" on BMS (Business Management Solutions) which ultimately proved to be a hatstand made of jello. Very few, outside of management and that product team, believed in it.&nbsp; Another significant WTF for Borland was, If you plan to be the Application LIFECYCLE Management company - why divest yourself (for a paltry $27m) of two of the world's major AppDev toolsets (Delphi and JBuilder). You've just removed the feeder market and upsell opportunity into your ALM business. Finally, and internal WTF to get off my chest, on what planet does the IT department belong as a subdivision of the HR department?<br /><br />Borland will live on in the hearts of many of us who knew what she used to be. I think I left Borland a better place than I found it (as long as you don't look at the stock price ;), and I made some good friends. At the end of the day, there isn't much more you can ask from your tenure.<br />It is sad that today if you ask a typical Software Engineer if they know who Borland is, they'll respond "Who?" which typifies the company's slide into obscurity.<br /><br />I wish the best of luck to my former colleagues, who I'm sure, will be wondering what is to happen next. I also hope that the new owners, Micro Focus International (who?), have good fortune with their ALM drive. Perhaps the Borland name might live on as a brand for a suite of ALM products - who knows what they'll do.<br /><br />In the immortal words of Dr. Seuss "Don't cry because it's over. Smile because it happened."<br /><br /> ]]>
        
    </content>
</entry>

<entry>
    <title>PHP algorithms: Determining if an IP is within a specific range.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2009/04/php-algorithms-determining-if-an-ip-is-within-a-specific-range.html" />
    <id>tag:pgregg.com,2009:/blog//2.333</id>

    <published>2009-04-27T22:48:15Z</published>
    <updated>2009-06-03T20:27:31Z</updated>

    <summary><![CDATA[I spend a lot of time lurking in the #PHP channel (efnet and freenode, please - no flamewars) and this topic is a commonly asked one that usually gets a simplified answer in the form of using strpos(), or at best an ip2long() in a greater than and less than answer. Unfortunately although people usually understand that an IP address is simply an unsigned 32 bit integer, and is easily determined, usually with $_SERVER['REMOTE_ADDR'], where the real challenge is - is in specifying the range within which they wish to check that IP address.&nbsp; IP ranges are usually specified in...]]></summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="PHP" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="algorithms" label="algorithms" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="cidr" label="CIDR" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="ipaddress" label="IP address" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="php" label="PHP" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[I spend a lot of time lurking in the #PHP channel (efnet and freenode, please - no flamewars) and this topic is a commonly asked one that usually gets a simplified answer in the form of using strpos(), or at best an ip2long() in a greater than and less than answer.
<br />
<br />Unfortunately although people usually understand that an IP address is simply an unsigned 32 bit integer, and is easily determined, usually with $_SERVER['REMOTE_ADDR'], where the real challenge is - is in specifying the range within which they wish to check that IP address.&nbsp; IP ranges are usually specified in three common ways (in increasing complexity):
<br />
<br />
<blockquote>
  <ol>
    <li>Wildcard: 192.168.10.*</li>
    <li>Start-End range: 10.1.0.0-10.1.255.255</li>
    <li>CIDR*: 172.16.1.0/24</li>
  </ol>
</blockquote>
<blockquote>* Classless Inter-Domain Routing
  <br />
</blockquote>
<p>The Wildcard method, or "classy", allows you to work at Class A (10.*.*.*), Class B (172.16.*.*) or Class C (192.168.10.*) levels of granularity which is how we used to do things in the old days (before the Web decided to make the Internet popular).&nbsp; But, increasingly, this just isn't granular enough for practical purposes.
  <br />
  <br />Thus was born CIDR (yes, I'm skipping talking about Start-End ranges for now).&nbsp; CIDR brought about the concept that we really didn't need to break networks on 8, 16, 24 bit boundaries and we could be more granular by allowing the use of any number (from 2-30) to specify a range of networks.&nbsp; Details on why you can't use "31" is beyond the scope of this article.
  <br />
  <br />CIDR renamed the former Class A, B and C networks as /8, /16 and /24 respectively and reflects the left-most significant bits of the 32-bit IP address.&nbsp; Thus was born the ability to specify very specific IP ranges in the form a.b.c.d/xx.&nbsp;&nbsp; However, part of the problem with this is that although it concisely describes the network start and end, most normal mortal humans couldn't decipher it. CIDR addressing can also be specified in the form of a longer netmask, e.g. a.b.c.d/255.255.255.224
  <br />
  <br />Thus, the simplified form of Start IP - End IP was put in place for mere mortals and is typically used by those without a networking background.&nbsp; It also features heavily in consumer broadband routers and notably in Microsoft Windows DHCP server.
  <br />
  <br />So having explained how a range, and by inference, that a netmask is, how can we use this knowledge to help us in determining if an IP is within a range?
  <br />
  <br />What this article will attempt to do is guide you though the construction of algorithms to make the checking of IPs simpler.
  <br />
  <br />Logically, Method 1 (the Wildcard), can be easily converted to Method 2 (Start-End range) by using setting Start and End to the Wildcard string and replacing the "*" character with 0 for the Start and 255 for the End, thus for example, "192.168.10.*" becomes "192.168.10.0-192.168.10.255" which should (I hope) be obvious to everyone.
  <br />
  <br />We can then proceed to evaluate both Method1 and Method2 in the same way.&nbsp; In this we're simply going to use the PHP built in function ip2long() on all 3 values and perform a mathematical check for Start &lt;= IP &lt;= End.
  <br />
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(255, 128, 0);">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span style="color: rgb(0, 119, 0);">list(</span><span style="color: rgb(0, 0, 187);">$lower</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(0, 0, 187);">$upper</span><span style="color: rgb(0, 119, 0);">)&nbsp;=&nbsp;</span><span style="color: rgb(0, 0, 187);">explode</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(221, 0, 0);">'-'</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(0, 0, 187);">$range</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(0, 0, 187);">2</span><span style="color: rgb(0, 119, 0);">);
  <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$lower_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">ip2long</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lower</span><span style="color: rgb(0, 119, 0);">);
  <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$upper_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">ip2long</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$upper</span><span style="color: rgb(0, 119, 0);">);
  <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$ip_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">ip2long</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$ip</span><span style="color: rgb(0, 119, 0);">);
  <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;(&nbsp;(</span><span style="color: rgb(0, 0, 187);">$ip_dec</span><span style="color: rgb(0, 119, 0);">&gt;=</span><span style="color: rgb(0, 0, 187);">$lower_dec</span><span style="color: rgb(0, 119, 0);">)&nbsp;&amp;&amp;&nbsp;(</span><span style="color: rgb(0, 0, 187);">$ip_dec</span><span style="color: rgb(0, 119, 0);">&lt;=</span><span style="color: rgb(0, 0, 187);">$upper_dec</span><span style="color: rgb(0, 119, 0);">)&nbsp;);</span></span></code>
  <br />
</p>
<p>We have, however, a complicating factor here - PHP does not do unsigned integers (32 bit) - which would be necessary for this math to work properly.&nbsp; We can negate this by switing to floating point data types.&nbsp; PHP stores&nbsp; floating types as 64 bit and so will have no problem with IPv4 address space (note - this obviously isn't granular enough for 128bit IPv6 addressing). Therefore the simplest way to solve the Start &lt;= IP &lt;= End problem with IPs and floating point numers is the following piece of code:
</p>
<p><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; </span><span style="color: rgb(0, 0, 187);">$lower_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;(float)</span><span style="color: rgb(0, 0, 187);">sprintf</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(221, 0, 0);">"%u"</span><span style="color: rgb(0, 119, 0);">,</span><span style="color: rgb(0, 0, 187);">ip2long</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lower</span><span style="color: rgb(0, 119, 0);">));
  <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$upper_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;(float)</span><span style="color: rgb(0, 0, 187);">sprintf</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(221, 0, 0);">"%u"</span><span style="color: rgb(0, 119, 0);">,</span><span style="color: rgb(0, 0, 187);">ip2long</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$upper</span><span style="color: rgb(0, 119, 0);">));
  <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$ip_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;(float)</span><span style="color: rgb(0, 0, 187);">sprintf</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(221, 0, 0);">"%u"</span><span style="color: rgb(0, 119, 0);">,</span><span style="color: rgb(0, 0, 187);">ip2long</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$ip</span><span style="color: rgb(0, 119, 0);">));
  <br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;(&nbsp;(</span><span style="color: rgb(0, 0, 187);">$ip_dec</span><span style="color: rgb(0, 119, 0);">&gt;=</span><span style="color: rgb(0, 0, 187);">$lower_dec</span><span style="color: rgb(0, 119, 0);">)&nbsp;&amp;&amp;&nbsp;(</span><span style="color: rgb(0, 0, 187);">$ip_dec</span><span style="color: rgb(0, 119, 0);">&lt;=</span><span style="color: rgb(0, 0, 187);">$upper_dec</span><span style="color: rgb(0, 119, 0);">)&nbsp;);
  <br /></span></span></code>
</p>
<p>Next we have the challenge of handing the CIDR netmasks. What we could do is to take a CIDR format IPaddress/netmask and calculate the Start and End IPs of that block and proceed as before - but that would be no fun - and would mean I haven't really taught anything through this article.&nbsp;
  <br />
  <br />The method we're going to use here is how all the world's Internet routers determine if a destination IP is in a specific CIDR address space. And we're going to get down and dirty with bitmasks and logical bitwise operators.
  <br />
  <br />So using a real world example, my webserver IP 80.76.201.37 and the netblock within which it resides is 80.76.201.32/27, how does this all work?
  <br />
  <br />Well the /27 indicates that the first 27 bits of the IP address are the same network and IP address in that network (range) will have those same identical first 27 bits.&nbsp; Bits 28-32 are variable and allow 5 bits of variation.&nbsp; If you know your binary, then this means 32 possible IPs. (However with routing, you can't use the bottom and top IP from any range as these are special and mean the network and broadcast addresses respectively. [This is also why a /31 isn't much use (except for PPP links) as you can't use the 2 addresses that space gives you]).
  <br />
  <br />So thinking logically, bitwise, if I take my IP address and the CIDR spec, then all I have to do is check that the first 27 bits all match and I'm good. Correct.&nbsp; So how would we do this in PHP? Sound's simple, lets just use PHP's bitwise logical AND operator: &amp;
  <br />Again, correct.&nbsp;
  <br />
  <br />In order to do this we need to convert 27 into what 27 really means - a 32 bit number of 27 ones and 5 zeros in binary (which is what 255.255.255.224 really looks like).
  <br />
  <br />In pseudo-code you could then do if (IP &amp; BITMASK) == (RANGE &amp; BITMASK) then all is good and you know that the IP is within the range.
  <br />
  <br />Visualising this using our real IP address (using the very handy unix tool ipcalc):
  <br />
</p>
<blockquote>
  <pre><font face="courier new,courier,monospace"><small>Address:   <strong>80.76.201.37</strong>    <strong>01010000.01001100.11001001.001</strong>00101
Netmask:   255.255.255.224 11111111.11111111.11111111.11100000
Wildcard:  0.0.0.31        00000000.00000000.00000000.000<strong>11111</strong>

Network:   <strong>80.76.201.32/27</strong> <strong>01010000.01001100.11001001.001</strong>00000
HostMin:   80.76.201.33    01010000.01001100.11001001.00100001
HostMax:   80.76.201.62    01010000.01001100.11001001.00111110
Broadcast: <strong>80.76.201.63</strong>    <strong>01010000.01001100.11001001.00111111</strong>
Hosts/Net: 30</small></font></pre>
</blockquote>You can see this in the Wildcard line of 0.0.0.31, and the Network ORed with Wildcard results in the Broadcast address: 80.76.201.63.
<br />
<br />Knowing this, then the IP address ANDed with the Network address will result in the same value as the Range ANDed with the Network address and so can be used as a comparison for an IP residing within that broadcast range.
<br />
<br />How can we work out this Network address in PHP, again we have two strategies, one is to so a simple substr() and take the left most significant bits of the range and then simply pad out to the right with 0s.&nbsp; Or we can do some math with "NOT of 2 to the power of (32-range) - 1". Thus for our value /27 this gives us the decimal value 31, NOTed results in (65536-31)&nbsp; (representational in the bit form - PHP will see it as a negative integer, but we don't need to worry about that).
<br />
<br />I'm sure by now, your screaming for some code (and if you stuck around this long, you really deserve it).
<br />
<br />Code to manipulate a range/netmask into a broadcast address, using math, assuming:
<br />
<blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$ip&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(221, 0, 0);">"80.76.201.37"</span><span style="color: rgb(0, 119, 0);">;</span></span></code>
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span><span style="color: rgb(0, 0, 187);">$range&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(221, 0, 0);">"80.76.201.32"</span><span style="color: rgb(0, 119, 0);">;</span></span></code>
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span><span style="color: rgb(0, 0, 187);">$netmask&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">27</span><span style="color: rgb(0, 119, 0);">;</span></span></code>
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code>
</blockquote> We can convert the IPs to long integers using ip2long (denoted by variable_dec - dec being short for decimal):
<br />
<blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span><span style="color: rgb(0, 0, 187);">$range_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">ip2long</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$range</span><span style="color: rgb(0, 119, 0);">);</span></span></code>
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span><span style="color: rgb(0, 0, 187);">$ip_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">ip2long</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$ip</span><span style="color: rgb(0, 119, 0);">);</span></span></code>
  <br />
</blockquote>This gives us the basis of our math, we now just need to work out the broadcast address.
<br />
<br />Strategy 1 using str_pad to create a string by padding with 1s and 0s.
<br />
<blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code><code><span style="color: rgb(0, 0, 0);"></span></code><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(255, 128, 0);"></span><span style="color: rgb(0, 0, 187);">$netmask_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">= </span></span></code><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span><span style="color: rgb(0, 0, 187);">bindec( str_pad</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(221, 0, 0);">''</span><span style="color: rgb(0, 119, 0);">, </span><span style="color: rgb(0, 0, 187);"></span><span style="color: rgb(0, 119, 0);"></span><span style="color: rgb(0, 0, 187);">$netmask</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(221, 0, 0);">'1'</span><span style="color: rgb(0, 119, 0);">)</span></span></code>
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; . </span><span style="color: rgb(0, 0, 187);">str_pad</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(221, 0, 0);">''</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(0, 0, 187);">32</span><span style="color: rgb(0, 119, 0);">-</span><span style="color: rgb(0, 0, 187);">$netmask</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(221, 0, 0);">'0'</span><span style="color: rgb(0, 119, 0);">) );</span></span></code>
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code>
</blockquote>We can achieve the same result though mathematics by NOTtin the wildcard value. This is our Strategy 2:
<br />
<blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(255, 128, 0);"></span><span style="color: rgb(0, 0, 187);">$wildcard_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">pow</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">2</span><span style="color: rgb(0, 119, 0);">,&nbsp;(</span><span style="color: rgb(0, 0, 187);">32</span><span style="color: rgb(0, 119, 0);">-</span><span style="color: rgb(0, 0, 187);">$netmask</span><span style="color: rgb(0, 119, 0);">))&nbsp;-&nbsp;</span><span style="color: rgb(0, 0, 187);">1</span><span style="color: rgb(0, 119, 0);">;</span></span></code>
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$netmask_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;~&nbsp;</span><span style="color: rgb(0, 0, 187);">$wildcard_dec</span><span style="color: rgb(0, 119, 0);">;</span></span></code>
  <br />
</blockquote>Once we know the netmask address (in decimal) as we have here, we can know that, if by ANDing this with the original IP to check results against the Range ANDed with the Netmask, then the IP is within the range defined by the range/mask.
<br />
<br />This can be checked easily with:
<br />
<blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);">return&nbsp;((</span><span style="color: rgb(0, 0, 187);">$ip_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">&amp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$netmask_dec</span><span style="color: rgb(0, 119, 0);">)&nbsp;==&nbsp;(</span><span style="color: rgb(0, 0, 187);">$range_dec&nbsp;</span><span style="color: rgb(0, 119, 0);">&amp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$netmask_dec</span><span style="color: rgb(0, 119, 0);">));</span></span></code>
  <br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code>
</blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code>I have pulled all of this logic together in a easily included file to provide a single function called ip_in_range($ip, $range) in which $ip is the IP address you want to validate and $range is a any of the above formats, Wildcard, Start-End addressing or CIDR.&nbsp; The function will return a simple TRUE or FALSE if the IP is in that range.
<br />
<br />The source code to the all-in function is available here:
<br />
<blockquote><a href="http://pgregg.com/projects/php/ip_in_range/ip_in_range.phps">http://pgregg.com/projects/php/ip_in_range/ip_in_range.phps</a>
  <br />
</blockquote>With an example run (and source code):
<br />
<blockquote><a href="http://pgregg.com/projects/php/ip_in_range/test.php">http://pgregg.com/projects/php/ip_in_range/test.php</a>
  <br />
</blockquote>
<p>I hope this article has been educational, please feel free to leave comments or feedback.
  <br />
</p>
<p>Update: There have been questions about PHP's signed integers and my use of bit operations in the code.&nbsp;&nbsp; It is important to recognise that when dealing with signed or unsigned 32 bit integers purely as bit patterns for masking with a netmask or broadcast address pattern - the fact that a number (128.0.0.0 or above) really is negative, doesn't have any impact on the validity of the result.&nbsp; The only impact to not having signed 32 bit integers is in the Start-End range check (example 2 above: 10.1.0.0-10.1.255.255) where a range spanning the switch from positive to negative would be catastrophic to the check.&nbsp; We can safely work around that problem by using floating point numbers as we are only doing &lt;= and &gt;= comparisons and not attempting any bitwise operators (which don't work on floats).
</p>]]>
        
    </content>
</entry>

<entry>
    <title>Caught red-handed.</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2009/04/caught-red-handed.html" />
    <id>tag:pgregg.com,2009:/blog//2.332</id>

    <published>2009-04-26T10:49:35Z</published>
    <updated>2009-04-26T23:42:01Z</updated>

    <summary>Only two hours after he was supposed to be asleep... Only light source was the Nintendo DS....</summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="General" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Photography" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="shane" label="shane" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[Only two hours after he was supposed to be asleep...<br /><br /><span class="mt-enclosure mt-enclosure-image" style="display: inline;"><img alt="IMG_7460_crop_500w.jpg" src="http://pgregg.com/blog/images/IMG_7460_crop_500w.jpg" class="mt-image-none" style="" width="500" height="311" /></span><br /> <div><br />Only light source was the Nintendo DS.<br /></div>]]>
        
    </content>
</entry>

<entry>
    <title>tail -# file, in PHP</title>
    <link rel="alternate" type="text/html" href="http://pgregg.com/blog/2009/04/tail---file-in-php.html" />
    <id>tag:pgregg.com,2009:/blog//2.331</id>

    <published>2009-04-23T20:05:48Z</published>
    <updated>2009-06-03T20:18:17Z</updated>

    <summary><![CDATA[I read Kevin's Read Line from File article today and thought I would add some sample code I wrote a while back to address a similar task in PHP - How to perform the equivalent of "tail -# filename" in PHP.A typical task in some applications such as a shoutbox or a simple log file viewer is how to extract the last 'n' lines from a (very) large file - efficiently.&nbsp; There are several approaches one can take to solving the problem (as always in programming there are many ways to skin the proverbial cat) - taking the easy way...]]></summary>
    <author>
        <name>Paul Gregg</name>
        <uri>http://pgregg.com</uri>
    </author>
    
        <category term="PHP" scheme="http://www.sixapart.com/ns/types#category" />
    
    <category term="algorithms" label="algorithms" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="file" label="file" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="php" label="PHP" scheme="http://www.sixapart.com/ns/types#tag" />
    <category term="tail" label="tail" scheme="http://www.sixapart.com/ns/types#tag" />
    
    <content type="html" xml:lang="en" xml:base="http://pgregg.com/blog/">
        <![CDATA[I read Kevin's <a href="http://phpro.org/examples/Read-Line-From-File.html">Read Line from File</a> article today and thought I would add some sample code I wrote a while back to address a similar task in PHP - How to perform the equivalent of "tail -# filename" in PHP.<br /><br />A typical task in some applications such as a shoutbox or a simple log file viewer is how to extract the last 'n' lines from a (very) large file - efficiently.&nbsp; There are several approaches one can take to solving the problem (as always in programming there are many ways to skin the proverbial cat) - taking the easy way out and shelling out to run the tail command; or reading the file line by line keeping the last 'n' lines in memory with a queue.<br /><br />However, here I will demonstrate a fairly tuned method of seeking to the end of the file and stepping back to read sufficient lines to the end of the file. If insufficient lines are returned, it incrementally looks back further in the file until it either can look no further, or sufficient lines are returned. <br /><br />Assumptions need to be made in order to tune the algorithm for the competing challenges of :<br /><ul><li>Reading enough lines in</li><li>as few reads as possible</li></ul>My approach to this is to provide an approximation to the average size of a line:<code> <span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$linelength</span></span></code><br />Multiply by 'n': <code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$linecount</span></span></code><br />and we have the byte <code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$offset </span></span></code>from the end of the file that we will seek to to begin reading.<br /><br />A check we need to do right here is that we haven't offset to before the beginning of the file, and so we have to override <code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$offset </span></span></code>to match the file size.<br /><br />Also as I'm being over-cautious, I'm going to tell it to offset <code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$linecount + 1</span></span></code> - The main reason for this is by seeking to a specific byte location in the file, we would have to be very lucky to land on the first character of a new line - therefore we must perform a fgets() and throw away that result.<br /><br />Typically, I want it to be able to read 'n' lines forward from the offset given, however if that proves insufficient, I'm going to grow the offset by 10% and try again. I also want to make it so that the algorithm is better able to tune itself if we grossly underestimate what <code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$linelength </span></span></code>should be.&nbsp; In order to do this, we're going to track the string length of each line we do get back and adjust the offset accordingly.<br /><br />In our example, lets try reading the last 10 lines from Apache's access_log<br /><br />So let's look at the code so far, nothing interesting, we're just prepping for the interesting stuff:<br /><blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$linecount&nbsp;&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">10</span><span style="color: rgb(0, 119, 0);">;&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;Number&nbsp;of&nbsp;lines&nbsp;we&nbsp;want&nbsp;to&nbsp;read<br /></span><span style="color: rgb(0, 0, 187);">$linelength&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">160</span><span style="color: rgb(0, 119, 0);">;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;Apache's&nbsp;logs&nbsp;are&nbsp;typically&nbsp;~200+&nbsp;chars<br />//&nbsp;I've&nbsp;set&nbsp;this&nbsp;to&nbsp;&lt;&nbsp;200&nbsp;to&nbsp;show&nbsp;the&nbsp;dynamic&nbsp;nature&nbsp;of&nbsp;the&nbsp;algorithm<br />//&nbsp;offset&nbsp;correction.<br /></span><span style="color: rgb(0, 0, 187);">$file&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(221, 0, 0);">'/usr/local/apache2/logs/access_log.demo'</span><span style="color: rgb(0, 119, 0);">;<br /></span><span style="color: rgb(0, 0, 187);">$fsize&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">filesize</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$file</span><span style="color: rgb(0, 119, 0);">);</span></span></code><br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code><br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span><span style="color: rgb(255, 128, 0);">//&nbsp;check&nbsp;if&nbsp;file&nbsp;is&nbsp;smaller&nbsp;than&nbsp;possible&nbsp;max&nbsp;lines</span></span></code><br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(255, 128, 0);"></span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;(</span><span style="color: rgb(0, 0, 187);">$linecount</span><span style="color: rgb(0, 119, 0);">+</span><span style="color: rgb(0, 0, 187);">1</span><span style="color: rgb(0, 119, 0);">)&nbsp;*&nbsp;</span><span style="color: rgb(0, 0, 187);">$linelength</span><span style="color: rgb(0, 119, 0);">;</span></span></code><br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);">if&nbsp;(</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">&gt;&nbsp;</span><span style="color: rgb(0, 0, 187);">$fsize</span><span style="color: rgb(0, 119, 0);">)&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">$fsize</span><span style="color: rgb(0, 119, 0);">;</span></span></code><br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code></blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code>Next up we're going to open the file and using our method of seeking to the end of the file, less our offset, here is the meat of our routine:<br /><br /><blockquote><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);"></span></span></code><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 0, 187);">$fp&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">fopen</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$file</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(221, 0, 0);">'r'</span><span style="color: rgb(0, 119, 0);">);<br />if&nbsp;(</span><span style="color: rgb(0, 0, 187);">$fp&nbsp;</span><span style="color: rgb(0, 119, 0);">===&nbsp;</span><span style="color: rgb(0, 0, 187);">false</span><span style="color: rgb(0, 119, 0);">)&nbsp;exit;<br /><br /></span><span style="color: rgb(0, 0, 187);">$lines&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;array();&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;array&nbsp;to&nbsp;store&nbsp;the&nbsp;lines&nbsp;we&nbsp;read.<br /><br /></span><span style="color: rgb(0, 0, 187);">$readloop&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">true</span><span style="color: rgb(0, 119, 0);">;<br />while(</span><span style="color: rgb(0, 0, 187);">$readloop</span><span style="color: rgb(0, 119, 0);">)&nbsp;{<br />&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;we&nbsp;will&nbsp;finish&nbsp;reading&nbsp;when&nbsp;we&nbsp;have&nbsp;read&nbsp;$linecount&nbsp;lines,&nbsp;or&nbsp;the&nbsp;file<br />&nbsp;&nbsp;//&nbsp;just&nbsp;doesn't&nbsp;have&nbsp;$linecount&nbsp;lines<br /><br />&nbsp;&nbsp;//&nbsp;seek&nbsp;to&nbsp;$offset&nbsp;bytes&nbsp;from&nbsp;the&nbsp;end&nbsp;of&nbsp;the&nbsp;file<br />&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">fseek</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$fp</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(0, 0, 187);">0&nbsp;</span><span style="color: rgb(0, 119, 0);">-&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(0, 0, 187);">SEEK_END</span><span style="color: rgb(0, 119, 0);">);<br /><br /></span></span></code><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);">&nbsp; </span><span style="color: rgb(255, 128, 0);">//&nbsp;discard&nbsp;the&nbsp;first&nbsp;line&nbsp;as&nbsp;it&nbsp;won't&nbsp;be&nbsp;a&nbsp;complete&nbsp;line<br />&nbsp;&nbsp;//&nbsp;unless&nbsp;we're&nbsp;right&nbsp;at&nbsp;the&nbsp;start&nbsp;of&nbsp;the&nbsp;file<br />&nbsp;&nbsp;</span><span style="color: rgb(0, 119, 0);">if&nbsp;(</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">!=&nbsp;</span><span style="color: rgb(0, 0, 187);">$fsize</span><span style="color: rgb(0, 119, 0);">)&nbsp;</span><span style="color: rgb(0, 0, 187);">fgets</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$fp</span><span style="color: rgb(0, 119, 0);">);</span></span></code><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span><span style="color: rgb(0, 119, 0);"><br /><br />&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;tally&nbsp;of&nbsp;the&nbsp;number&nbsp;of&nbsp;bytes&nbsp;in&nbsp;each&nbsp;line&nbsp;we&nbsp;read<br />&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$linesize&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">0</span><span style="color: rgb(0, 119, 0);">;<br /><br />&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;read&nbsp;from&nbsp;here&nbsp;till&nbsp;the&nbsp;end&nbsp;of&nbsp;the&nbsp;file&nbsp;and&nbsp;remember&nbsp;each&nbsp;line<br />&nbsp;&nbsp;</span><span style="color: rgb(0, 119, 0);">while(</span><span style="color: rgb(0, 0, 187);">$line&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">fgets</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$fp</span><span style="color: rgb(0, 119, 0);">))&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">array_push</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lines</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(0, 0, 187);">$line</span><span style="color: rgb(0, 119, 0);">);<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$linesize&nbsp;</span><span style="color: rgb(0, 119, 0);">+=&nbsp;</span><span style="color: rgb(0, 0, 187);">strlen</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$line</span><span style="color: rgb(0, 119, 0);">);&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;total&nbsp;up&nbsp;the&nbsp;char&nbsp;count<br /><br />&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;if&nbsp;we've&nbsp;been&nbsp;able&nbsp;to&nbsp;get&nbsp;more&nbsp;lines&nbsp;than&nbsp;we&nbsp;need<br />&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;lose&nbsp;the&nbsp;first&nbsp;entry&nbsp;in&nbsp;the&nbsp;queue<br />&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;Logically&nbsp;we&nbsp;should&nbsp;decrement&nbsp;$linesize&nbsp;too,&nbsp;but&nbsp;if&nbsp;we<br />&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;hit&nbsp;the&nbsp;magic&nbsp;number&nbsp;of&nbsp;lines,&nbsp;we&nbsp;are&nbsp;never&nbsp;going&nbsp;to&nbsp;use&nbsp;it<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 119, 0);">if&nbsp;(</span><span style="color: rgb(0, 0, 187);">count</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lines</span><span style="color: rgb(0, 119, 0);">)&nbsp;&gt;&nbsp;</span><span style="color: rgb(0, 0, 187);">$linecount</span><span style="color: rgb(0, 119, 0);">)&nbsp;</span><span style="color: rgb(0, 0, 187);">array_shift</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lines</span><span style="color: rgb(0, 119, 0);">);<br />&nbsp;&nbsp;}<br /><br />&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;We&nbsp;have&nbsp;now&nbsp;read&nbsp;all&nbsp;the&nbsp;lines&nbsp;from&nbsp;$offset&nbsp;until&nbsp;the&nbsp;end&nbsp;of&nbsp;the&nbsp;file<br />&nbsp;&nbsp;</span><span style="color: rgb(0, 119, 0);">if&nbsp;(</span><span style="color: rgb(0, 0, 187);">count</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lines</span><span style="color: rgb(0, 119, 0);">)&nbsp;==&nbsp;</span><span style="color: rgb(0, 0, 187);">$linecount</span><span style="color: rgb(0, 119, 0);">)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;perfect&nbsp;-&nbsp;have&nbsp;enough&nbsp;lines,&nbsp;can&nbsp;exit&nbsp;the&nbsp;loop<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$readloop&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">false</span><span style="color: rgb(0, 119, 0);">;<br />&nbsp;&nbsp;}&nbsp;elseif&nbsp;(</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">&gt;=&nbsp;</span><span style="color: rgb(0, 0, 187);">$fsize</span><span style="color: rgb(0, 119, 0);">)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;file&nbsp;is&nbsp;too&nbsp;small&nbsp;-&nbsp;nothing&nbsp;more&nbsp;we&nbsp;can&nbsp;do,&nbsp;we&nbsp;must&nbsp;exit&nbsp;the&nbsp;loop<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$readloop&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">false</span><span style="color: rgb(0, 119, 0);">;<br />&nbsp;&nbsp;}&nbsp;elseif&nbsp;(</span><span style="color: rgb(0, 0, 187);">count</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lines</span><span style="color: rgb(0, 119, 0);">)&nbsp;&lt;&nbsp;</span><span style="color: rgb(0, 0, 187);">$linecount</span><span style="color: rgb(0, 119, 0);">)&nbsp;{<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;try&nbsp;again&nbsp;with&nbsp;a&nbsp;bigger&nbsp;offset<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">intval</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">*&nbsp;</span><span style="color: rgb(0, 0, 187);">1.1</span><span style="color: rgb(0, 119, 0);">);&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;increase&nbsp;offset&nbsp;10%<br />&nbsp;&nbsp;&nbsp;&nbsp;//&nbsp;but&nbsp;also&nbsp;work&nbsp;out&nbsp;what&nbsp;the&nbsp;offset&nbsp;could&nbsp;be&nbsp;if&nbsp;based&nbsp;on&nbsp;the&nbsp;lines&nbsp;we&nbsp;saw<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset2&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">intval</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$linesize</span><span style="color: rgb(0, 119, 0);">/</span><span style="color: rgb(0, 0, 187);">count</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lines</span><span style="color: rgb(0, 119, 0);">)&nbsp;* </span>(<span style="color: rgb(0, 0, 187);">$linecount+1)</span><span style="color: rgb(0, 119, 0);">);<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;and&nbsp;if&nbsp;it&nbsp;is&nbsp;larger,&nbsp;then&nbsp;use&nbsp;that&nbsp;one&nbsp;instead&nbsp;(self-tuning)<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 119, 0);">if&nbsp;(</span><span style="color: rgb(0, 0, 187);">$offset2&nbsp;</span><span style="color: rgb(0, 119, 0);">&gt;&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset</span><span style="color: rgb(0, 119, 0);">)&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset2</span><span style="color: rgb(0, 119, 0);">;<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;Also&nbsp;remember&nbsp;we&nbsp;can't&nbsp;seek&nbsp;back&nbsp;past&nbsp;the&nbsp;start&nbsp;of&nbsp;the&nbsp;file<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 119, 0);">if&nbsp;(</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">&gt;&nbsp;</span><span style="color: rgb(0, 0, 187);">$fsize</span><span style="color: rgb(0, 119, 0);">)&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;</span><span style="color: rgb(0, 0, 187);">$fsize</span><span style="color: rgb(0, 119, 0);">;<br />&nbsp;&nbsp;&nbsp;&nbsp;echo&nbsp;</span><span style="color: rgb(221, 0, 0);">'Trying&nbsp;with&nbsp;a&nbsp;bigger&nbsp;offset:&nbsp;'</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(0, 0, 187);">$offset</span><span style="color: rgb(0, 119, 0);">,&nbsp;</span><span style="color: rgb(221, 0, 0);">"\n"</span><span style="color: rgb(0, 119, 0);">;<br /></span></span></code><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);">&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(255, 128, 0);">//&nbsp;and&nbsp;reset<br />&nbsp;&nbsp;&nbsp;&nbsp;</span><span style="color: rgb(0, 0, 187);">$lines&nbsp;</span><span style="color: rgb(0, 119, 0);">=&nbsp;array();<br /></span></span></code><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);">&nbsp; }<br />}<br /><br /></span><span style="color: rgb(255, 128, 0);">//&nbsp;Let's&nbsp;have&nbsp;a&nbsp;look&nbsp;at&nbsp;the&nbsp;lines&nbsp;we&nbsp;read.<br /></span><span style="color: rgb(0, 0, 187);">print_r</span><span style="color: rgb(0, 119, 0);">(</span><span style="color: rgb(0, 0, 187);">$lines</span><span style="color: rgb(0, 119, 0);">);</span></span></code><br /><code><span style="color: rgb(0, 0, 0);"><span style="color: rgb(0, 119, 0);"></span></span></code></blockquote><br /><br />At first glance it might seem line overkill for the task, however stepping through the code you can see the expected while loop with fgets() to read each line. The only thing we are doing at this stage is shifting the first line of the $lines array if we happen to read too many lines, and also tallying up how many characters we managed to read for each line.<br /><br />If we exit the while/fgets loop with the correct number of lines, then all is well, we can exit the main retry loop and we have the result in $lines.<br /><br />Where the code gets interesting is what we do if we don't achieve the required number of lines.&nbsp; The simple fix is to step back by a further 10% by increasing the offset and trying again, but remember we also counted up the number of bytes we read for each line we did get, so we can very simply obtain an average real-file line size by dividing this with the number of lines in our $lines array. This enables us to override the previous offset value to something larger, if indeed we were wildly off in our estimates.<br /><br />By adjusting our offset value and letting the loop repeat, the routine will try again and repeat until it succeeds or fails gracefully by the file not having sufficient lines, in which case it'll return what it could get.<br /><br />For the complete, working, source code please visit:<br /><blockquote><a href="http://pgregg.com/projects/php/code/tail-10.phps">http://pgregg.com/projects/php/code/tail-10.phps</a><br /></blockquote>Sample execution:<br /><blockquote><a href="http://pgregg.com/projects/php/code/tail-10.php">http://pgregg.com/projects/php/code/tail-10.php</a><br /></blockquote> <div><br /></div>]]>
        
    </content>
</entry>

</feed>

