9.3.1. Methods for Searching the Tree

$node->find_by_tag_name(tag [, ...])

Return node(s) for tags of the names listed. For example, to find all h1 and h2 nodes:

@headings = $root->find_by_tag_name('h1', 'h2');

$node->find_by_attribute(attribute, value)

Returns the node(s) with the given attribute set to the given value. For example, to find all nodes with class="blinking":

@blinkers = $root->find_by_attribute("class",
"blinking");

$node->look_down(...)

$node->look_up(...)

These two methods search $node and its children (and children's children, and so on) in the case of look_down, or its parent (and the parent's parent, and so on) in the case of look_up, looking for nodes that match whatever criteria you specify. The parameters are either attribute => value pairs (where the special attribute _tag represents the tag name), or a subroutine that is passed a current node and returns true to indicate that this node is of interest.

For example, to find all h2 nodes in the tree with class="blinking":

@blinkers = $root->look_down(_tag => 'h2', class => 'blinking');

We'll discuss look_down in greater detail later.

9.3.2. Attributes of a Node

$node->tag( )

The tag name string of this element. Example values: html, img, blockquote. Note that this is always lowercase.

$node->parent( )

This returns the node object that is the parent of this node. If $node is the root of the tree, $node->parent( ) will return undef.

$node->content_list( )

This returns the (potentially empty) list of nodes that are this node's children.

$node->attr(attributename)

This returns the value of the HTML attributename attribute for this element. If there is no such attribute for this element, this returns undef. For example: if $node is parsed from <img src="x1.jpg" alt="Looky!">, then $node->attr("src") will return the string x1.jpg.

Four more methods convert a tree or part of a tree into another format, such as HTML or text.

$node->as_HTML([ entities [, indent_char [, optional_end_tags ]]]);

Returns a string consisting of the node and its children as HTML. The entities parameter is a string containing characters that should be entity escaped (if empty, all potentially unsafe characters are encoded as entities; if you pass just <>&, just those characters will get encoded—a bare minimum for valid HTML). The indent_char parameter is a string used for indenting the HTML. The optional_end_tags parameter is a reference to a hash that has a true value for every key that is the name of a tag whose closing tag is optional. The most common value for this parameter is {} to force all tags to be closed:

$html = $node->as_HTML("", "", {});

For example, this will emit </li> tags for any li nodes under $node, even though </li> tags are technically optional, according to the HTML specification.

Using $node->as_HTML( ) with no parameters should be fine for most purposes.

$node->as_text( )

Returns a string consisting of all the text nodes from this element and its children.

$node->starttag([entities])

Returns the HTML for the start-tag for this node. The entities parameter is a string of characters to entity escape, as in the as_HTML( ) method; you can omit this. For example, if this node came from parsing <TD class=loud>Hooboy</TD>, then $node->starttag( ) returns <td class="loud">. Note that the original source text is not reproduced exactly, because insignificant differences, such as the capitalization of the tag name or attribute names, will have been discarded during parsing.

$node->endtag( )

Returns the HTML for the end-tag for this node. For example, if this node came from parsing <TD class=loud>Hooboy</TD>, then $node->endtag( ) returns </td>.

9.3.3. Traversing

$node->traverse(callbacks [, ignore_text]);

The traverse( ) method calls a callback before processing the children and again afterward. If the callbacks parameter is a single function reference, the same function is called before and after processing the children. If the callbacks parameter is an array reference, the first element is a reference to a function called before the children are processed, and the second element is similarly called after the children are processed, unless this node is a text segment or an element that is prototypically empty, such as br or hr. (This last quirk of the traverse( ) method is one of the reasons that I discourage its use.)

Callbacks get called with three values:

sub callback 
  my ($node, $startflag, $depth,
      $parent, $my_index) = @_;
  # ...
}

A callback can return any of the following values:

HTML::Element::OK (or any true value)

Continue traversing.

HTML::Element::PRUNE (or any false value)

Do not go into the children. The postorder callback is not called. (Ignored if returned by a postorder callback.)

HTML::Element::ABORT

Abort the traversal immediately.

HTML::Element::PRUNE_UP

Do not go into this node's children or into its parent node.

HTML::Element::PRUNE_SOFTLY

Do not go into the children, but do call this node's postorder callback.

For example, to extract text from a node but not go into table elements:

my $text;
sub text_no_tables {
  return if ref $_[0] && $_[0]->tag eq 'table';
  $text .= $_[0] unless ref $_[0];  # only append text nodex
  return 1;                         # all is copacetic
}

$root->traverse([\&text_no_tables]);

This prevents descent into the contents of tables, while accumulating the text nodes in $text.

It can be hard to think in terms of callbacks, though, and the multiplicity of return values and calling parameters you get with traverse( ) makes for confusing code, as you will likely note when you come across its use in existing programs that use HTML::TreeBuilder.

Instead, it's usually easier and clearer to simply write your own recursive subroutine, like this one:

my $text = '';
sub scan_for_non_table_text {
  my $element = $_[0];
  return if $element->tag eq 'table';   # prune!
  foreach my $child ($element->content_list) {
    if (ref $child) {  # it's an element
      scan_for_non_table_text($child);  # recurse!
    } else {           # it's a text node!
      $text .= $child;
    }
  }
  return;
}
scan_for_non_table_text($root);

Alternatively, implement it using a stack, doing the same work:

my $text = '';
my @stack = ($root);  # where to start
 
while (@stack) {
  my $node = shift @stack;
  next if ref $node and $node->tag eq 'table';  # skip tables
  if (ref $node) {
    unshift @stack, $node->content_list;        # add children
  } else {
    $text .= $node;                             # add text
  }
}

The while( ) loop version can be faster than the recursive version, but at the cost of being much less clear to people who are unfamiliar with this technique. If speed is a concern, you should always benchmark the two versions to make sure you really need the speedup and that the while( ) loop version actually delivers. The speed difference is sometimes insignificant. The manual page perldoc HTML::Element::traverse discusses writing more complex traverser routines, in the rare cases where you might find this necessary.