Introduction to the Standalone WWW Blast server

Index

• Introduction
• What's new in this revision?
• Installation of the Standalone WWW server
• Description of files in the distribution
• Configuration of BLAST databases
• PSI/PHI Blast notes
• Client/server versions for Entrez lookup and taxonomy reports
• Blast 2 sequences
• XML output
• Out of Frame BLASTX
• RPS Blast
• Description of tags for main BLAST input page
• Server configuration file and logfile
• How to debug WWW Blast programs

Introduction

This standalone WWW BLAST server suite of programs was designed similar to the regular NCBI BLAST server and such command-line NCBI BLAST programs like "blastall", "blastpgp", "rpsblast" and "megablast". It incorporates most features, which exist in NCBI BLAST programs and should be relatively easy to use. This server does not support any request queuing and load balancing. As soon as the user hits a "Search" button, BLAST starts immediately if entered information is valid. So, this server is not intended to handle large load, which may exist in public service. Such queueing and loadbalancing however may be implemented using such products as Load Sharing Facility - "LSF" from Platform Computing Corporation. Interface to "LSF" was implemented in NCBI, however this was not included in this suite. Standalone server assumes that users have their own BLAST or RPS-BLAST database(s), that should be searched and want to have a simple WWW interface to such search. It is STRONLY recommended that user has experience in installation and running standalone NCBI BLAST programs.

After files are uncompressed, server is ready to be used immediately. Any customizations to the program are welcomed and may be done by experienced programmers using source code, which is also provided.

Recompilation of the server executables require that programmer has compiled NCBI toolkit libraries. This toolkit can be downloaded from the NCBI FTP web site: ftp://ftp.ncbi.nih.gov
After the NCBI toolkit is compiled, user needs to edit the Makefile in the blast/Src directory, and define the NCBI_LIBDIR (or NCBI_ALTLIB, if toolkit was compiled with debugging symbols), and NCBI_INCDIR variables in the top 3 lines of the Makefile, so they point to the directories where the NCBI toolkit libraries and include files are located.

What's new in this revision?

Installation of the Standalone WWW server

After downloading the file wwwblast.Your_platform.tar.gz to your computer, place it into document directory of HTTPD server and uncompress it by

    gzip -d wwwblast.Your_platform.tar.gz
    tar -xvpf wwwblast.Your_platform.tar

Please note that parameter "p" in tar options is significant - it will preserve file access options stored in the distribution. Temporary directory for BLAST overview images (TmpGifs) should have 777 permission, and logfiles (wwwblast.log and psiblast.log) should have 666.

After you have uncompressed the distribution file, "blast" directory will be created. You can access sample BLAST HTML input forms using URLs:

• http://your_hostname/blast/blast.html
• http://your_hostname/blast/blast_cs.html
• http://your_hostname/blast/psiblast.html
• http://your_hostname/blast/psiblast_cs.html
• http://your_hostname/blast/megablast.html
• http://your_hostname/blast/megablast_cs.html
• http://your_hostname/blast/wblast2.html
• http://your_hostname/blast/wblast2_cs.html

Description of files in the distribution

• Root directory (./blast):

blast.cgi, blast_cs.cgi	- BLAST search start-up C-shell files
.nlmstmanrc	- Configuration file for the graphical overview image (do not edit!)
blast.html, blast_cs.html	- sample BLAST search input HTML forms
megablast.html, megablast_cs.html	- sample MEGABLAST search input HTML forms
rpsblast.html, rpsblast_cs.html	- sample RPS BLAST search input HTML forms
blast.rc	- Default configuration file for the WWW BLAST server
psiblast.rc	- Default configuration file for the PSI/PHI WWW BLAST server
psiblast.cgi, psiblast_cs.cgi	- PSI/PHI BLAST search start-up C-shell files
psiblast.html, psiblast_cs.html	- sample PSI/PHI BLAST search input HTML forms
psiblast.REAL, psiblast_cs.REAL	- Main PSI/PHI BLAST server executables
wblast2.html, wblast2_cs.html	- sample BLAST 2 sequences search input HTML forms
wblast2.REAL, wblast2_cs.REAL	- Main BLAST 2 sequences server executables
bl2bag.cgi	- CGI used to create 2 sequences alignment image on the fly
blast_form.map	- Auxiliary map file for the front BLAST image
nph-viewgif.cgi	- CGI program used to view and delete overview images
readme.html	- this documentation
wwwblast.log	- default logfile
psiblast.log	- default PSI/PHI Blast logfile
ncbi_blast.rc	- sample file for full NCBI set of databases

• ./data directory - matrices used in BLAST search
• ./db directory. - Files of test BLAST and RPS-BLAST databases.
• ./docs: - HTML pages used in sample BLAST search input pages
• ./images - images used in the sample BLAST search input pages
• ./Src - source directory for the WWW BLAST server.
• ./Src/XML - source directory for creation of files related to the XML output.
  • blstxml.asn - ASN.1 definition for Blast XML
  • blstxml.dtd - corresponding DTD
• ./TmpGifs - storage for temporary BLAST overview gif files

Configuration of BLAST databases

To set up databases for the standalone WWW BLAST server, it is necessary to follow these steps:

1. Put a file with concatenated FASTA entries in the "./db" directory
2. Run "formatdb" program, available from the NCBI ftp site to format the database.
3. Add name of the database into server configuration file
4. Add name of the database into (PSI/PHI) WWW BLAST search form

PSI/PHI Blast notes

Databases for the PSI/PHI Blast should always be created with formatdb using the "-o T" flag. Test database "test_aa_db" was created using this flag and database "test_na_db" was created without this flag.

If this distribution is installed not in the "/blast" directory under HTTPD documents root directory, than path to the distribution should be set by an environment variable WWW_ROOT_PATH in the file psiblast.cgi or psiblast_cs.cgi

Client/server version for Entrez lookup and taxonomy reports

XML output

Blast 2 sequences

RPS Blast

Databases for RPS blast are hardware dependent - for speed reasons. Hence they are different for big/little endian platforms.

To build RPS database it is necessary to follow the procedure explained in the file "README.rps", that comes with this distribution. There is a small RPS database available for testing. This database is a part of a real NCBI database used in CDD search page. Full NCBI database is available in platform-independent form from FTP site.

Out of Frame BLASTX

XML and tabular output are not yet implemented for the OOF alignments.

Description of tags for the main BLAST input page

This standalone server has a tag convention analogous to the regular NCBI BLAST server. Sample BLAST search forms may be changed to accommodate particular needs of the user in the custom search. Here is the list of these tags and their meaning. If some tag is missing from the search input page, it will take a default value. Exceptions are tags PROGRAM, DATALIB and SEQUENCE (or SEQFILE), that should always be set.

• PROGRAM - name of the BLAST program. Supported values include programs: blastn, blastp, blastx, tblastx and tblastn
• DATALIB - name of the database(s) to search. This implementation includes possibility to use multiple databases. To use multiple databases several "DATALIB" tags should be used on the page for example using checkboxes (look for example at Microbial Genomes Blast Databases BLAST at NCBI). Note, that all of these databases should be properly written in the server configuration file.
• SEQUENCE and SEQFILE - these tags used to pass sequence. First SEQUENCE tag is used for the input sequence. If it is missing, the SEQFILE tag is used instead.
• UNGAPPED_ALIGNMENT - default BLAST search is a gapped search; this tag, if set, will turn gapped alignment off.
• MAT_PARAM used to set 3 parameters at the same time. Value for this tag should be in format " " where mat_name - string name of the matrix (BLOSUM62, etc), d1 - integer for cost to open gap and d2 - cost to extend gap (-G and -E parameters in blastall respectably)
• GAP_OPEN - set value for cost to open gap - 0 or missing tag invoked default behavior
• GAP_EXTEND - set value for cost to extend gap - 0 or missing tag invoked default behavior
• X_DROPOFF - Dropoff (X) for blast extensions in bits (default if zero) (-y parameter in "blastpgp" program)
• GENETIC_CODE - Query Genetic code to use (for blastx only)
• THRESHOLD_2 - Threshold for extending hits in second pass in multipass model search
• MATRIX - Matrix (default is BLOSUM62) (-M in blastall)
• EXPECT - Expectation value (-e in blastall)
• NUM_OF_BITS - Number of bits to trigger gapping (-N in blastpgp)
• NCBI_GI - If formated database use SeqIds in the NCBI format this option will turn printing of gis together with accessions.
• FILTER - Multiple instances of values of this tag are concatenated and passed to the engine as "filter_string" ("L" for low complexity and "m" if filter should be set for lookup table only) - any letter will turn default filtering on - DUST for nucleotides and SEG for proteins (-F in blastall)
• DESCRIPTIONS - Number of one-line descriptions in the output (-v in blastall)
• ALIGNMENTS - Number of alignments to show (-b in blastall)
• COLOR_SCHEMA - Color schema to use in printing of alternative alignment. This option is valid only for blastp and blastn programs. If set - it will override option set by "ALIGNMENT_VIEW"
• TAX_BLAST - Print taxonomy reports. This option is valid only for client/server version of regular Blast
• XML_OUTPUT - Print XML Blast output. All other alignment view options will be disabled
• ENTREZ_QUERY - Limit search to results of Entrez query. Only for client/server version
• RPSBLAST - This tag with turn "blastp" or "blastx" search into RPS Blast search for the rps blast database.
• OOF_ALIGN -This flag if set to non-zero digit will turn on OOF alignment for "blastx" and will set frame shift penalty to this value.
• OTHER_ADVANCED - this tag allows to input string analogous to the command line parameters of blastall. Setting parameter in OTHER_ADVANCED tag will override all other settings of this parameter. Supported options include:
• -G gap open cost
• -E gap extend cost
• -q penalty for nucleotide mismatch
• -r reward for nucleotide match
• -e expect value
• -W wordsize
• -v Number of descriptions to print
• -b Number of alignments to show
• -K Number of best hits from a region to keep
• -Y effective search space
• ALIGNMENT_VIEW - will set type of alignment to show. Available options include:
• 0 - Pairwise
• 1 - query-anchored with identities
• 2 - query-anchored without identities
• 3 - flat query-anchored with identities
• 4 - flat query-anchored without identities
• 7 - BLAST XML (equivalent to setting the XML_OUTPUT tag)
• 9 - Hit Table - to produce tabular output
• OVERVIEW - used to turn on or off printing of alignment overview image
• WWW_BLAST_TYPE - a special tag to distinguish different BLAST search types. See the description of a configuration file.

Server configuration file and logfile

Default configuration file is "blast.rc" and logfile "wwwblast.log". Setting tag WWW_BLAST_TYPE to specific value may change these names. This is useful if few different search input pages use the same CGI search engine, but significantly different by content and priorities. A sample configuration file comes with this distribution:

How to debug WWW Blast programs

1. Add line "setenv DEBUG_COMMAND_LINE TRUE" into the *.cgi file (uncomment it)
2. Run search that results in the problem - this should create a file "/tmp/__web.in" in the "/tmp" directory.
3. Set all necessary environment variables on the command line (BLASTDB at least) and run from command-line: "blast.REAL < /tmp/__web.in"

dbx blast.REAL core

and then use command "where" to print stack.