Python Code Examples

1
Python Code Examples
2
Word Spotting

• import sys
• fname1 "c\Python Course\ex1.txt"
• for line in open(fname1,'r').readlines()
• for word in line.split()
• if word.endswith('ing')
• print word

3
Creating a Dictionary of First Names

• def createNameDict()
• dictNameFileopen('project/dictionaries/names.txt
  ','r')
• dictContentdictNameFile.read() read all the
  file
• dictWordsdictContent.split(",") return a
  list with the words
• nameDict initialize a dictionary
• for word in dictWords
• nameDictword.strip()" " enters each word to
  the dctionary.
• return nameDict

4
Computing Accuracy Results I

• anfiles.py
• Program to analyze the results of speaker
  identification.
• Illustrates Python dictionarys
• import string, glob, sys
• def main()
• read correct file and test file
• fname1 sys.argv1
• fname2 sys.argv2
• text1 open(fname1,'r').read()
• text1 string.lower(text1)
• words1 string.split(text1)
• correct_len len(words1)
• text2 open(fname2,'r').read()

5
Computing Accuracy Results II

• construct a dictionary of correct results
• correct
• for w in words1
• correctw 1
• for i in range(correct_len)
• in_count 0
• portion2 words2i1
• for w in portion2
• if correct.get(w,0) gt 0
• in_count1
• accuracy float(in_count)/float(len(portion2
  ))
• print "5d, 5d,.2f" (len(portion2),
  in_count, accuracy)
• if __name__ '__main__' main()

6
Word Histograms

• import sre, string
• pattern sre.compile( r'a-zA-Z' )
• def countwords(text)
• dict
• try
• iterator pattern.finditer(text)
• for match in iterator
• word match.group()
• try
• dictword dictword 1
• except KeyError
• dictword 1
• except sre.error
• pass triggers when first index goes to
  -1, terminates loop.

7
Word Histograms

• items
• for word in dict.keys()
• items.append( (dictword, word) )
• items.sort()
• items.reverse()
• return items
• if run as a script, count words in stdin.
• if __name__ "__main__"
• import sys
• x countwords( sys.stdin.read() )
• s map(str, x)
• t string.joinfields(s, "\n")
• print t

8
Extracting People Names and Company Names

• import string, sre, glob, sys
• def createNameDict()
• dictNameFileopen('names.txt','r')
• dictContentdictNameFile.read() read all the
  file
• dictWordsdictContent.split(",") return a
  list with the words
• nameDict initialize a dictionary
• for word in dictWords
• nameDictword.strip()" " enters each word to
  the dctionary.
• return nameDict
• def main()
• read file
• fname1 sys.argv1
• text1 open(fname1,'r').read()
• namesDic createNameDict()

9
Extracting People Names and Company Names

• r'(corpCORPCorpltdL
  tdLTDincIncINCcorporationCorporationCORPORA
  TIONgmbhGMBHagAGsaSA)'
• r'(\.?)')
• pattern1 sre.compile( r'(A-Z\w\s.-)2,
  4' )
• Companies
• capitalWordssre.finditer(pattern,text1)
• for match in capitalWords
• CapSeq match.group()
• print CapSeq
• People
• capitalWords1sre.finditer(pattern1,text1)
  
• for match in capitalWords1
• wordListmatch.group().split()
• check name in names dictionary
• if namesDic.has_key(wordList0.strip())
• print match.group()

10
NLTK

• NLTK defines a basic infrastructure that can be
  used to build NLP programs in Python. It
  provides
• Basic classes for representing data relevant to
  natural language processing.
• Standard interfaces for performing tasks, such as
  tokenization, tagging, and parsing.
• Standard implementations for each task, which can
  be combined to solve complex problems.
• Extensive documentation, including tutorials and
  reference documentation.

11
RE Show

• gtgtgt from nltk.util import re_show
• gtgtgt string """
• ... Its probably worth paying a premium for
  funds that invest in markets
• ... that are partially closed to foreign
  investors, such as South Korea, ...
• ... """
• gtgtgt re_show(t..., string)
• Its probably worth paying a premium for
  funds that invest in markets that are
  partially closed to foreign investors, such
  as South Korea, ...
• gtgtgt

12
Classes in Python
13
Defining Classes

• gtgtgt class SimpleClass
• ... def __init__(self, initial_value)
• ... self.data initial_value
• ... def set(self, value)
• ... self.data value
• ... def get(self)
• ... print self.data
• ...
• gtgtgt x SimpleClass(4)

14
Inheritance

• B is a subclass of A
• gtgtgt class B(A)
• ... def __init__(self)
• SimpleTokenizer implements the interface of
  TokenizerI
• gtgtgt class SimpleTokenizer(TokenizerI)
• ... def tokenize(self, str)
• ... words str.split()
• ... return Token(wordsi,
  Location(i))
• ... for i in
  range(len(words))

15
Inheritance Example

• class point
• def __init__(self, x0, y0)
• self.x, self.y x, y
• class cartesian(point)
• def distanceToOrigin(self)
• return floor(sqrt(self.x2 self.y2))
• class manhattan(point)
• def distanceToOrigin(self)
• return self.x self.y

16
Sets
17
Sets in Python

• The sets module provides classes for constructing
  and manipulating unordered collections of unique
  elements. Common uses include
• membership testing,
• removing duplicates from a sequence,
• and computing standard math operations on sets
  such as intersection, union, difference, and
  symmetric difference.
• Like other collections, sets support x in set,
  len(set), and for x in set. Being an unordered
  collection, sets do not record element position
  or order of insertion. Accordingly, sets do not
  support indexing, slicing, or other sequence-like
  behavior.

18
Some Details about Implementation

• Most set applications use the Set class which
  provides every set method except for __hash__().
  For advanced applications requiring a hash
  method, the ImmutableSet class adds a __hash__()
  method but omits methods which alter the contents
  of the set.
• The set classes are implemented using
  dictionaries. As a result, sets cannot contain
  mutable elements such as lists or dictionaries.
• However, they can contain immutable collections
  such as tuples or instances of ImmutableSet.
• For convenience in implementing sets of sets,
  inner sets are automatically converted to
  immutable form, for example, Set(Set('dog'))
  is transformed to Set(ImmutableSet('dog')).

19
Set Operations
20
Operations not for ImmutableSet
21
Set Examples

• gtgtgt from sets import Set
• gtgtgt engineers Set('John', 'Jane', 'Jack',
  'Janice')
• gtgtgt programmers Set('Jack', 'Sam', 'Susan',
  'Janice')
• gtgtgt managers Set('Jane', 'Jack', 'Susan',
  'Zack')
• gtgtgt employees engineers programmers
  managers union
• gtgtgt engineering_management engineers managers
  intersection
• gtgtgt fulltime_management managers - engineers -
  programmers difference
• gtgtgt engineers.add('Marvin')
  add element
• gtgtgt print engineers
• Set('Jane', 'Marvin', 'Janice', 'John', 'Jack')
• gtgtgt employees.issuperset(engineers)
  superset test
• False

22
Set Examples

• gtgtgt employees.union_update(engineers)
  update from another set
• gtgtgt employees.issuperset(engineers)
• True
• gtgtgt for group in engineers, programmers,
  managers, employees
• ... group.discard('Susan')
  unconditionally remove element
• ... print group
• ...
• Set('Jane', 'Marvin', 'Janice', 'John', 'Jack')
• Set('Janice', 'Jack', 'Sam')
• Set('Jane', 'Zack', 'Jack')
• Set('Jack', 'Sam', 'Jane', 'Marvin', 'Janice',
  'John', 'Zack')

23
Google API

• Get it from http//sourceforge.net/projects/pygoog
  le/
• A Python wrapper for the Google web API. Allows
  you to do Google searches, retrieve pages from
  the Google cache, and ask Google for spelling
  suggestions.

24
Utilizing the Google API - I

• import sys
• import string
• import codecs
• import google
• print 'lthtml xmlns"http//www.w3.org/1999/xhtml"
  xmllang"en"gt'
• print 'ltheadgt'
• print ' lttitlegtGoogle with Pythonlt/titlegt'
• print 'lt/headgt'
• print 'ltbodygt'
• print 'lth1gtGoogle with Pythonlt/h1gt'
• google.LICENSE_KEY 'YOUR GOOGLE LICENSE KEY'
• sys.stdout codecs.lookup('utf-8')-1(sys.stdout
  )
• query Your Query"
• data google.doGoogleSearch(query)

25
Utilizing the Google API - II

• print 'ltpgtltstronggt1-10 of "' query '" total
  results for '
• print str(data.meta.estimatedTotalResultsCount)
  'lt/stronggtlt/pgt'
• for result in data.results
• title result.title
• title title.replace('ltbgt', 'ltstronggt')
• title title.replace('lt/bgt', 'lt/stronggt')
• snippet result.snippet
• snippet snippet.replace('ltbgt','ltstronggt')
• snippet snippet.replace('lt/bgt','lt/stronggt')
• snippet snippet.replace('ltbrgt','ltbr /gt')
• print 'lth2gtlta href"' result.URL '"gt'
  title 'lt/agtlt/h2gt'
• print 'ltpgt' snippet 'lt/pgt'
• print 'lt/bodygt
• print 'lt/htmlgt'

26
Yahoo API

• http//pysearch.sourceforge.net/
• http//python.codezoo.com/pub/component/4193?categ
  ory198
• This project implements a Python API for the
  Yahoo Search Webservices API. pYsearch is an OO
  abstraction of the web services, with emphasis on
  ease of use and extensibility.

27
URLLIB

• This module provides a high-level interface for
  fetching data across the World Wide Web.
• In particular, the urlopen() function is similar
  to the built-in function open(), but accepts
  Universal Resource Locators (URLs) instead of
  filenames.
• Some restrictions apply -- it can only open URLs
  for reading, and no seek operations are
  available.

28
Urllib Syntax

• Use http//www.someproxy.com3128 for http
  proxying
• proxies 'http' 'http//www.someproxy.com3128'
  
• filehandle urllib.urlopen(some_url,
  proxiesproxies)
• Don't use any proxies
• filehandle urllib.urlopen(some_url, proxies)

29
URLLIB Examples

• Here is an example session that uses the "GET"
  method to retrieve a URL containing parameters
• gtgtgt import urllib
• gtgtgt params urllib.urlencode('spam' 1, 'eggs'
  2, 'bacon' 0)
• gtgtgt f urllib.urlopen("http//www.musi-cal.com/cg
  i-bin/query?s" params)
• gtgtgt print f.read()
• The following example uses the "POST" method
  instead
• gtgtgt import urllib
• gtgtgt params urllib.urlencode('spam' 1, 'eggs'
  2, 'bacon' 0)
• gtgtgt f urllib.urlopen("http//www.musi-cal.com/cg
  i-bin/query", params)
• gtgtgt print f.read()

30
What is a Proxy

• A proxy server is a computer that offers a
  computer network service to allow clients to make
  indirect network connections to other network
  services.
• A client connects to the proxy server, then
  requests a connection, file, or other resource
  available on a different server.
• The proxy provides the resource either by
  connecting to the specified server or by serving
  it from a cache.
• In some cases, the proxy may alter the client's
  request or the server's response for various
  purposes.
• A proxy server can also serve as a firewall.