Search

Wednesday, January 23rd, 2008. Filed under ColdFusion

I recently had the need to search a web directory’s application files for a particular string. Not as a customer-facing function, just as a one-off development need.

Well, CFEclipse (as truly awesome a tool as it is) doesn’t have a way to search multiple files for the same string simultaneously, and while Home Site does have this feature (”Extended Find / Extended Replace”), if you use it on anything but the smallest directories, it consistently times out and needs to be restarted again. So below is a little utility I wrote to recursively search a given directory for a given string. Its basically just two Java objects wrapped in a filtered CFDIRECTORY loop.

“But Tyler, why did you use Java when you could have just used CFFILE?” you say. Well, for me there’s now been several projects wherein CFFILE has considerably underperformed compared to its Java counterparts, and with the buffered reader object specifically, I can pass in fairly decent size files without ColdFusion choking on them. And its fast. Blazing fast. My browser almost caught fire (but I guess that’s the price you pay for efficiency). So nowadays, if I need a function that CFFILE would normally handle (at least in regards to reading and writing), I first consider the Java alternatives.

By the way, there is significant mixing of logic and presentation here, so be warned if you need to avert your eyes. Anyway, HTH.

<!--- the string to search for --->
<cfset request.searchstring = "thestring" />
<!--- the path to look in --->
<cfset request.startingpath = "/var/www" />
<!--- should the search be recursive. careful on this one. --->
<cfset request.recurse = false>
<cfset request.filecount = 0 />
<cfset request.linecount = 0 />
<cfset request.fileshown = 0 />    

<cfdirectory name="files"
    action="list"
    directory="#request.startingpath#"
    recurse="#request.recurse#"
    filter="*.cfm|*.cfc|*.cfml|*.js|*.css|*.htm|*.html"
    sort="directory asc, name asc" />    

<html>
<head>
    <style>
        * {
            font-size: 12px;
            font-family: 'courier new';
        }
        table {
            border: 0;
            padding: 0;
            border-collapse: collapse;
        }
        th { text-align: left; }
        th, td { padding: 0 15px 0 5px; }
        .even { background: #dadada; }
        .filename { font-weight: bold; }
    </style>
</head>    

<body>    

    <table>
        <tr>
            <th colspan="3">directory/filename</th>
        </tr>
        <tr>
            <th> </th>
            <th>num</th>
            <th>line of code</th>
        </tr>    

        <cfloop query="files">
            <!--- exclude this file from the search --->
            <cfif files.name NEQ listlast(cgi.SCRIPT_NAME,"/")>    

                <!--- set the source file to be read --->
                <cfset variables.sourcefile = files.directory & "\" & files.name />
                <cfset variables.linenumber = 1 />    

                <!--- instantiate the necessary java objects --->
                <cfset variables.filereader = createObject("java","java.io.FileReader").init(variables.sourcefile) />
                <cfset variables.bufferedreader = createObject("java","java.io.BufferedReader").init(variables.filereader) />    

                <!--- set the current line to the first available line in the file --->
                <cfset variables.currentline = variables.bufferedreader.readLine() />    

                <!--- beginning looping over the file line by line --->
                <cfloop condition="isDefined(' variables.currentline')">    

                    <!--- check the current line for the given search string --->
                    <cfif variables.currentline CONTAINS request.searchstring >    

                        <cfif request.fileshown EQ 0>
                            <!--- stack multiple results under a single file name --->
                            <tr valign="top" <cfif request.filecount MOD 2 EQ 0> class="even" </cfif> >
                                    <td colspan="3" class="filename"><cfoutput>#variables.sourcefile#</cfoutput></td>
                            </tr>   
                            <cfset request.fileshown = 1 />
                        </cfif>    

                        <!--- list the line number and line code containing the search string --->
                        <tr valign="top" <cfif request.filecount MOD 2 EQ 0> class="even" </cfif> >
                                <td> </td>
                                <cfoutput>
                                    <td align="right">#variables.linenumber#</td>
                                    <td>#htmleditformat(variables.currentline)#</td>
                                </cfoutput>
                        </tr>    

                        <cfset request.linecount = request.linecount + 1 />
                    </cfif>    

                    <!--- try to get the next line in the file. --->
                    <cfset variables.currentline = variables.bufferedreader.readLine() />
                    <cfset variables.linenumber = variables.linenumber + 1 />
                </cfloop>    

                <!--- close the buffered reader object --->
                <cfset variables.bufferedreader.close() />
                <cfif request.fileshown EQ 1>
                    <cfset request.fileshown = 0 />
                    <cfset request.filecount = request.filecount + 1>
                </cfif>
            </cfif>
        </cfloop>    

        <!--- list the line number and line code containing the search string --->
        <tr valign="top" <cfif request.filecount MOD 2 EQ 0> class="even" </cfif> >
            <th colspan="3">
                <cfoutput>
                    #request.linecount# matching lines in #request.filecount# files. #files.recordcount# files searched.
                </cfoutput>
            </th>
        </tr>
    </table>    

</body>
</html>

THIS.mappings

Saturday, January 5th, 2008. Filed under ColdFusion

I’ve always been a bit perturbed at the CFIMPORT tag, both because you have to use the tag on every page in which you’ll call the prefixed custom tag, including pages that are called with CFINCLUDE (will save that rant for another time), and also because the taglib attribute cannot take a variable. To illustrate, you cannot do this:

<!--- this does not work --->
<cfset request.tagpath = "/var/www/tagsdirectory/" />
<cfimport prefix="t" taglib="#request.tagpath#" />

As you probably know, you must declare the custom tag path in the cfadmin, then call the CFIMPORT tag using the name you assigned to the path in the admin. Like so:

<cfimport prefix="t" taglib="/tagpath" />

That is until now. In ColdFusion 8 aka Scorpio aka SomeOtherCoolName, you can now create application-specific paths for your custom tags. You do so by creating a structure called THIS.mappings in your Application.cfc, then adding keys and values to this structure referencing the tag path name and the tag path directory, respectively. Like so:

<cfcomponent displayname="Application">
<!--- application-specific custom tag paths --->
<cfset this.mappings = structnew() />
<cfset this.mappings["/tagpath"] = "/var/www/tagsdirectory/" />

Note the required forward slash preceeding the tag name you assign. Now just call the CFIMPORT tag like normal at the top of your page:

<cfimport prefix="t" taglib="/tagpath" />

Bada bing bada boom, dynamic tag paths. HTH.

–>

Tutorial:

Tuesday, December 25th, 2007. Filed under Ubuntu

So in the continuing pursuit of setting up the preferred web development environment for myself on my home network, I installed Subversion (SVN) on my Ubuntu Desktop 7.10 “CLAMP” (ColdFusion + LAMP) server. Here’s the general process I wanted in place:

1. Run the SVN repository directly on my local web server, blocking anonymous access, and checking out my projects via the secure shell protocol (svn+ssh://).

2. Check out said projects from the repository onto my primary machine where I develop using Eclipse and the Subclipse plug-in. (I also use the CFEclipse and Aptana plug-ins. Props on two excellent products! )

3. Commit the changes to the repository on my Web Server, which would then auto-publish the latest copy of my source to the web server’s web root.

4. Drink and be merry.

Having SVN automatically post the latest version of my source directly to the web root proved to be a bit of a challenge for me as I’m basically a newbie to both Linux and the process of setting up SVN. This effort and the bit of research involved is the focus of this post. This post also assumes you’re running a Linux variant.

Advice Disclaimer: Before I start, I would like to make note that this setup is for my home network and sandbox, not a professional production environment. I would not suggest this configuration for such an instance. Perhaps it could work for moving source from a development area to a staging/debug area as some have suggested, but I would rather not “auto-publish” directly to a customer-facing application. Unless such functionality was tightly controlled, in my estimation there seems too much opportunity for problems to arise. End disclaimer.

There are essentially two different ways to have SVN automatically post your content to your web root, and both of them center around using SVN’s built-in hooks, specifically the post-commit hook. For more info on SVN hooks go here, but in essence they allow you to run specific scripts at various key points in the repository usage process. The post-commit script, as I’m sure you guessed by the name, is ran after a commit has successfully completed, and can execute whatever command or string of commands you’d like. This script can be basically anything, but the most common examples I’ve listed as “Options” following the walk-through on the post-commit script itself.

Firstly, the post-commit (and other scripts) can be found in the “hooks” folder of your repository location in the form “post-commit.tmpl”.

Step 1. Copy post-commit.tmpl into the same directory as just “post-commit”.

Step 2. Open the file, commenting out any uncommented lines of code with a hash symbol (#), like for example, the line at the bottom of the current SVN installation post-commit that tries to run an email notification with every commit. Save your changes.

Step 3. Give the script the proper permissions. Permissions seem to be the number one area in which people have problems running their post-commit scripts. When your script runs, it will do so as the SVN user, not you. That SVN user will need create/delete rights in the web root. I handled this by creating a SVN user group in Ubuntu which includes the SVN user as well as the Apache user: www-data. You may prefer other solutions. Like, for example, writing and compiling a small C program, and then just giving that program the rights to do what you want.

Also, the script being run on post-commit will need SVN repository permissions, so if you are using the standard SVN protocol, you’ll need to ensure that the appropriate user is listed in the SVN passwd file (and that the passwd file is configured in the svnserve.conf file). If you are connecting via SSH, you could create an additional user account and limit its priviledges to just this process. For general help setting up SVN on Ubuntu (my operating system of choice), check out this excellent guide on Ubuntu.com.

Step 4. Choose either option A or option B.

Option A.

The SVN export. This is the cleanest, and arguably the most “correct” method. However, it is also the heaviest in terms of processor load, memory usage, and time (if your site/application is of any but the smallest of sizes). The SVN export is as it sounds - a complete export of your most current version of source files, images, everything you have in version control. Its primary function is for the distribution of source for production. To utilize this function, you would again edit the post-commit file, and directly under the hashbang (the first line of code denoted by the “#!” character combination), add this line:

/usr/bin/svn export svn://localhost/repository /var/www --username somesvnuser --password theirpassword

Note the full path preceding the svn command as there is no internal path variable within SVN. This is intentional so as to avoid assumption-based errors in file management. Also note that we are designating the user to be the specific svn user which we have given the correct repository permissions (based either on the SVN passwd file, or for SVN+SSH, the system user accounts).

Though the SVN export is obviously a necessary function in version control, it does not work for my purposes here as I’m looking to post short incremental changes to my code base, and am not interested in overwriting and re-exporting my entire project every time I commit a change.

Continue to step 5.

Option B.

The SVN update. This method involves treating the web root as another working copy previously checked out from your repository, which is continually and automatically updated upon each successful commit. The key difference between an update and a full export are that the update only re-fetches the files which have changed since the last update. For my usage, this is _much_ more efficient than the previous option, and in the end was the method I chose. Here’s how to go this route: Remember that post-commit file? Edit it again and add and save this line, instead of the export line in the previous option:

/usr/bin/svn update /var/www --username somesvnuser --password theirpassword

Again, note the full path on the SVN command, and the specific user designation.

Now, before this script can be successfully ran, the working copy needs to be checked out into the web root. Do so in a terminal window with:

$ svn co svn://localhost/repository /var/www --username somesvnuser ---password theirpassword

Also, there is a catch: Whenever you (or a script) checks out or updates a working copy, SVN automatically creates some hidden “.svn” folders and files in the root of your project, this case being your web root. You may not notice them but they’re there. And in so existing are a security risk as they may contain information you’re likely to want to hide from public access. However, if you’re running Apache, this is easily handled with a few lines added to the httpd.conf file disallowing the access to those .svn folders and files. This may minimally increase Apache’s workload, but will definitely provide much appreciated convenience to you:

<DirectoryMatch "^/.*/.svn/">
    Order deny,allow
    Deny from all
</DirectoryMatch>

After you’ve added and saved this line, restart Apache:

$ /etc/init.d/apache2 restart

Continue to step 5.

Step 5. Make the post-commit script an executable application in a terminal window:

$ chmod +x post-commit

Step 6. If you’re using Eclipse with the Subclipse plug-in like I am, and you try to commit a change to the repository without your script having everything it needs to perform its task (like I did), Eclipse may hang on you, and you will get no feedback on the error. So, I suggest running the script itself in the terminal window first to get that much needed feedback. Run it without sudo btw. The script must have the permissions it needs to run on its own.

$ /your/repository/hooks/post-commit

If all goes well, you’re on your way. Any problems, please see the note above about permissions, as this is the most common issue in regard to running SVN scripts. Go browse the web root to ensure it contains the latest version of your project.

Step 7. Open Eclipse, make a change in the code, and try to commit. If it doesn’t hang, you’re good to go. If it does, go back to the terminal for feedback and re-examine your script and its permissions. Good luck!

Step 8. Say “Woo-hoo!”.

Supplemental: An interesting auto-publish option: Rsync. In researching this topic, I came across what I thought was an innovative take on the auto-publishing topic. Lincolnloop.com posts a process whereby the web server’s working copy is actually stored outside of the web root, and the post-commit script runs a shell command that rsync’s the web server’s working copy with another one in the web root, only it uses a separate text file containing names of folders and files that should not be transferred, cleaning resolving the issue of publishing sensitive information.

And that’s about all I have to say about that. Hope this helps someone. Peace.

–>