FilterPlugin < TWiki

TWiki Web>FilterPlugin (23 Oct 2019, TWikiAdminUser) (raw view)
---+!! <nop>%TOPIC%
<!--
   Contributions to this plugin are appreciated. Please update the plugin page at
   http://twiki.org/cgi-bin/view/Plugins/FilterPlugin or provide feedback at
   http://twiki.org/cgi-bin/view/Plugins/FilterPluginDev.
   If you are a TWiki contributor please update the plugin in the SVN repository.
-->
%SHORTDESCRIPTION%
<div style="float:right; background-color:#eeeeee; margin:0 0 20px 20px; padding: 0 10px 0 10px; width:380px">
%TOC{title="Page contents"}%
</div>

---++ Description

This plugin allows to substitute and extract information from content by using regular expressions. There are three different types of new functions:
   1 FORMATLIST: maniplulate a list of items; it is highly configurable to 
     define what constitutes a list and how to extract items from it
   1 SUBST, STARTSUBST/STOPSUBST: substiture a pattern in a chunk of text
   1 EXTRACT, STARTEXTRACT/STOPEXTRACT: extract a pattern from a text

While the START-STOP versions of SUBST and EXTRACT work on inline text, the normal versions process a source topic before including it into the current one.

---++ Syntax Rules

---+++ SUBST

*Syntax*: =%<nop>SUBST{topic="..." ...}%= 

Insert a topic by processing its content.

   * =topic="..."=: name of the topic text to be processed 
   * =text="..."=: text to be processed (has got higher precedence than 'topic')
   * =pattern="..."=: pattern to be extracted or substituted 
   * =format="..."=: format expression or pattern substitute 
   * =header="..."=: header string prepended to output
   * =footer="..."=: footer string appended to output
   * =limit="&lt;n>"= maximum number of occurences to extract or substitute counted from the start of the text (defaults to =100000= aka all hits) 
   * =skip="&lt;n>"= skip the first n occurences
   * =exclude="..."=: skip occurences that match this regular expression
   * sort="on,off,alpha,num" order of the formatted items (default "off")
   * =expand="on,off"=: toggle expansion of TWiki markup before filtering (defaults to =on=) 

---+++ STARTSUBST, STOPSUBST

*Syntax*:
<verbatim>
%STARTSUBST{...}% 
... 
%STOPSUBST%
</verbatim>

Substitute text given inline. see [[#SUBST][SUBST]].

---+++ EXTRACT

*Syntax*: =%<nop>EXTRACT{topic="..."  ...}%= 

Extract text from a topic. see [[#SUBST][SUBST]].

---+++ STARTEXTRACT, STOPEXTRACT

*Syntax*:
<verbatim>
%STARTEXTRACT{...}% 
... 
%STOPEXTRACT%
</verbatim>

Extract content given inline. see [[#SUBST][SUBST]].


---+++ FORMATLIST

*Syntax*: =%<nop>FORMATLIST{"&lt;list>" ...}%=

Formats a list of items. The &lt;list> argument is separated into items by using a split expression; each item is matched against a pattern and then formatted using a format string while being separated by a separator string; the result is prefixed with a header and appended with a footer in case the list is not empty.
   * &lt;list>: the list 
   * split="...": the split expression (default ",")
   * pattern="...": pattern applied to each item (default "\s(.*)\s")
   * format="...": the format string for each item (default "$1")
   * header="...": header string
   * footer="...": footer string
   * separator="...": string to be inserted between list items
   * limit="...": max number of items to be taken out of the list (default "-1")
   * skip="...": number of list items to skip, not adding them to the result
   * sort="on,off,alpha,num" order of the formatted items (default "off")
   * reverse="on,off": reverse the sortion of the list
   * unique="on,off": remove dupplicates from the list
   * exclude="...": remove list items that match this regular expression

The pattern string shall group matching substrings in the list item to which you can refer to by using $1, $2, ... in the format string. Any format string (=format=, =header=, =footer=) may contain variables =$percnt$=, =$nop=, =$dollar= and =$n=. The variable =$index= referse to the position number within the list being formatted; using =$count= in the footer or header argument refers to the total number of list elements.

---+++ MAKEINDEX

*Syntax*: =%<nop>MAKEINDEX{"&lt;list>" ...}%=

Formats a list into a multi-column index like in <nop>MediaWiki's category topcis. MAKEINDEX insert capitals as headlines to groups of sorted items. It will try to balance all columns equally, and keep track of breaks to prevent "schusterkinder", that is avoid isolated headlines at the bottom of a column.

Parameters:
   * &lt;list>: the list of items
   * split="...": the split expression to separate the &lt;list> into items (default ",")
   * pattern="...": pattern applied to each item (default "(.*)")
   * cols="...": maximum number of cols to split the list into
   * format="...": format of each list item (default "$item")
   * sort="on/off": sort the list (default "on")
   * unique="on/off": removed duplicates (default "off")
   * exclude="...": pattern to check against items in the list to be excluded
   * reverse="on/off": reverse the list (default "off")
   * header="...": format string to prepend to the result
   * footer="..." format string to be appended to the result

Like in FORMATLIST the =format= parameter can make use of $1, $2, ... variables to match the groupings defined in the =pattern= argument (like in =pattern="(.*);(.*);(.*)"=) . The first matched grouping $1 will be used as the $item to sort the list.

---++ Examples

---+++ Secure Html

%STARTSUBST{pattern="<(a href=\"javascript:.*?)>(.*?)</a>" format="&lt;$1&gt;$2&lt;/a&gt;"}%
<a href="javascript:window.alert('Pop me up');">Pop me up</a>
%STOPSUBST%

---+++ Format Comments

| *Date* | *Author* | *Headline* |
%EXTRACT{topic="%TOPIC%" expand="off" pattern=".div class=\"text\">.*?[\r\n]+(.*?)[\r\n]+(?:.*?[\r\n]+)+?-- (.*?) on (.*?)[\r\n]+" format="| $3 | $2 | $1 ... |$n"}%

%STARTINCLUDE%
<div class="text"><div class="comment">
This is a first comment.
</div><div class="posted">
-- TWiki:Main.MichaelDaum on 22 Aug 2005
</div></div>

<div class="text"><div class="comment">
This is a second comment.
</div><div class="posted">
-- TWiki:Main.MichaelDaum on 22 Aug 2005
</div></div>

---+++ Extract table data

| *Pos* | *Description* | *Hours* | 
|  1 | onsite troubleshooting |  3 | 
|  2 | normalizing data to new format |  10 |
|  3 | testing server performace |  5 |
%STOPINCLUDE%

%EXTRACT{topic="%TOPIC%" 
  expand="off" 
  pattern="\|\s*(.*?)\s*\|\s*(.*?)\s*\|\s*(.*?)\s*\|" 
  format="   * it took $3 hours $2$n"
  skip="1"
}%

---+++ MAKEINDEX example

Compare with [[http://en.wikipedia.org/wiki/Category:Philosophy_articles_needing_attention][Philosophy articles needing attention]]

%MAKEINDEX{
  "Absolute (philosophy), Accident (philosophy), Actualism, Talk:Adam Weishaupt, Alphabet of human thought, Alterity, Analytic philosophy, Analytic-synthetic distinction, Apologism, Bundle theory, Categories (Stoic), Causal chain, Causality, Coherentism, Conscience, Context principle, Contextualism, Cosmology, De dicto and de re, Dialectical monism, Difference (philosophy), Direct reference theory, Discourse ethics, Dualism, Emergentism, Essence, Ethical naturalism, Exemplification, Existentialism, Fatalism, French materialism, Futilitarianism, Hermeneutics, Hypokeimenon, Identity and change, Idolon tribus, Immanent evaluation, Indeterminacy (Philosophy), Individual, Inherence, Kennisbank Filosofie Nederland, Lazy Reason, Mike Lesser, Libertarianism (metaphysics), Logicism, Mad pain and Martian pain, Materialism, Meaning of life, Metakosmia, Metaphysical naturalism, Milesian school, Mind, Monism, Moral imperative, Multiplicity (philosophy), Mystical philosophy of antiquity, Nature (philosophy), Neomodernism, New England Transcendentalists, Nominalism, Non-archimedean time, Non-rigid designator, Object (philosophy), Ontic, Ontological reductionism, Phenomenology, Philosophical realism, Philosophical skepticism, Philosophy, Pluralism (philosophy), Post-structuralism, Postmodern philosophy, Preferentialism Present (time), Problem of universals, Process philosophy, Rational Animal, Rationalist movement, Relativism, Self (philosophy), Solipsism, Species (metaphysics), Specters of Marx, Substance theory, Talk:The Art of Being Right, Truth-value link, Universal (metaphysics), Utilitarianism, Value judgment, World riddle" 
  cols="3"
  format="[[http://en.wikipedia.org/wiki/$item][$item]]"
}%

---++ Plugin Installation Instructions

   * Download the ZIP file from twiki.org (see below)
   * Unzip it in your twiki installation directory. Content: 
     | *File:* | *Description:* |
   | ==data/TWiki/FilterPlugin.txt== |  |
   | ==lib/TWiki/Plugins/FilterPlugin/Core.pm== |  |
   | ==lib/TWiki/Plugins/FilterPlugin.pm== |  |

   * Visit =configure= in your TWiki installation, and enable the plugin in the {Plugins} section.

---++ Plugin Info

<!-- provided for compatibility only
   * Set SHORTDESCRIPTION = Substitute and extract information from content by using regular expressions
-->
|  Plugin Author: | TWiki:Main.MichaelDaum |
|  Copyright: | &copy; 2005-2008, Michael Daum http://michaeldaumconsulting.com <br /> &copy; 2005-2010, TWiki:TWiki/TWikiContributor |
|  License: | GPL ([[http://www.gnu.org/copyleft/gpl.html][GNU General Public License]]) |
|  Plugin Version: | 2010-07-25 |
|  Change History: | <!-- versions below in reverse order -->&nbsp; |
|  2010-07-25: | TWikibug:Item6530 - doc improvements |
|  2010-04-29: | TWikibug:Item6433 - doc improvements |
|  03 Jul 2008: | sorting a list _before_, not _after_, formatting it in FORMATLIST |
|  08 May 2008: | added 'text' parameter to SUBST and EXTRACT; \
                  fixed SUBST as it was pretty useless before |
|  07 Dec 2007: | added MAKEINDEX, added lazy compilation |
|  14 Sep 2007: | added sorting for EXTRACT and SUBST |
|  02 May 2007: | using registerTagHandler() as far as possible; \
                  enhanced parameters to EXCTRACT and SUBST |
|  05 Feb 2007: | fixed escapes in format strings; \
                  added better default value for max number of hits to prevent deep recursions \
                  on bad regexpressions |
|  22 Jan 2007: | fixed SUBST, added skip parameter to FORMATLIST |
|  18 Dec 2006: | using registerTagHandler for FORMATLIST |
|  13 Oct 2006: | fixed =limit= parameter in FORMATLIST |
|  31 Aug 2006: | added NO_PREFS_IN_TOPIC |
|  15 Aug 2006: | added =use strict;= and fixed revealed errors |
|  14 Feb 2006: | moved in FORMATLIST from the TWiki:Plugins/NatSkinPlugin;\
                  added escape variables to format strings |
|  06 Dec 2005: | fixed SUBST not to cut off the rest of the text |
|  09 Nov 2005: | fixed deep recursion using =expand="on"= |
|  22 Aug 2005: | Initial version; added =expand= toggle |
|  TWiki Dependency: | $TWiki::Plugins::VERSION 1.024 |
|  CPAN Dependencies: | none |
|  Other Dependencies: | none |
|  Perl Version: | 5.005 |
|  TWiki:Plugins/Benchmark: | %SYSTEMWEB%.GoodStyle nn%, %SYSTEMWEB%.FormattedSearch nn%, %TOPIC% nn% |
|  Plugin Home: | http://TWiki.org/cgi-bin/view/Plugins/%TOPIC% |
|  Feedback: | http://TWiki.org/cgi-bin/view/Plugins/%TOPIC%Dev |
|  Appraisal: | http://TWiki.org/cgi-bin/view/Plugins/%TOPIC%Appraisal |

__Related Topics:__ %SYSTEMWEB%.TWikiPlugins, %SYSTEMWEB%.DeveloperDocumentationCategory, %SYSTEMWEB%.AdminDocumentationCategory, %SYSTEMWEB%.TWikiPreferences
Topic revision: r3 - 23 Oct 2019 - TWikiAdminUser
Account
- Log In
- Register User
Edit
Attach
Copyright © 1999-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
Note: Please contribute updates to this topic on TWiki.org at TWiki:TWiki.FilterPlugin.