Virtual Environment Setup:

As I had already installed python-virtualenv, I've jumpted directly to create a new virual environment:

$mkdir virtual
$cd virtual
$virtualenv virtual_planetparse
The PYTHONDONTWRITEBYTECODE environment variable is not compatible with setuptools. Either use --distribute or unset PYTHONDONTWRITEBYTECODE.

First problem arises. Setting PYTHONDONTWRITEBYTECODE="" works it around

$PYTHONDONTWRITEBYTECODE="" virtualenv virtual_planetparse
New python executable in virtual_planetparse/bin/python
Installing setuptools............done.
Installing pip...............done.
$ source virtual_planetparse/bin/activate
(virtual_planetparse) $ pip install beautifulsoup4
[...]
Successfully installed beautifulsoup4
Cleaning up...
(virtual_planetparse) $ pip install html5lib
Successfully installed html5lib
Cleaning up...

Code

Link to the code

It works as explained in the comments. it gets the html of the site with urllub2.urlopen(). Then it parses it using BeautifulSoup, and select(). the syntax used to select the desired tags is the following:

".blog-entry-author > a" # The tag "a" (link) inside the tag of class (note the point meaning class) "blog-entry-author"

This is the whole code:

 1 #!/usr/bin/env python
 2 
 3 # Assignment: Get the titles and authors of all the blogs feeded
 4 # at http://planet.fedoraproject.org.
 5 #
 6 # Student: Josep Caselles
 7 # Course: #dgplug Summer Training Course
 8 # Date: 14/07/2013
 9 
10 from sys import exit
11 from urllib2 import urlopen
12 from bs4 import BeautifulSoup
13 
14 URL_CONSTANT = "http://planet.fedoraproject.org"
15 
16 def print_blog_info ():
17 
18     """
19     This method will use BeautifulSoup to parse the content of the given url
20     and extract from it the desired content. With select() method from
21     BeautifulSoup you can get all tags given it's class, id, or any other
22     attribute. for a complete reference, see http://tinyurl.com/nn4m7hg.
23 
24     Steps made:
25         1- Fetch the whole html with urllib2 urlopen()
26         2- "Soupe" it with BeautifulSoup
27         3- Select the desired tag's content
28         4- print accordingly
29 
30     """
31 
32     try:
33         html_doc = urlopen (URL_CONSTANT)
34 
35     except:
36         exit("\nError: Something is wrong with http://planet.fedoraproject.org"
37              " or your internet connection\n")
38 
39     html_souped = BeautifulSoup (html_doc)
40     html_doc.close()
41 
42     z = 0
43 
44     for x, y in zip(html_souped.select(".blog-entry-author > a"),
45                     html_souped.select(".blog-entry-title > a")):
46 
47         z += 1
48 
49         print """
50 Blog Entry n. %.2i:
51 -----------------
52 
53 Tile: '%s'
54 Author: %s
55         """ % (z, y.string, x.string)
56 
57 
58 if __name__ == "__main__":
59     print_blog_info ()
60     exit(0)

Code

Link to Code at GitHub

  1 #!/usr/bin/env python
  2 
  3 # Assignment: Retrieve from http://download.finance.yahoo.com  the share value
  4 # of a given NASDAQ company's symbol.
  5 #
  6 #To achieve this, we need the urllib2 module.
  7 #
  8 
  9 import sys
 10 import urllib2
 11 
 12 
 13 class  data_retriever (object):
 14 
 15     """
 16     This class will serve as base class for retrieving content from urls.
 17     Splitting this in two classes may be dumb, but it will serve as
 18     an hineritance exercise as well.
 19 
 20     """
 21 
 22     def __init__ (self):
 23          self.content = None # Content is where all data will be stored
 24 
 25     def return_data ():
 26 
 27         """
 28         Will retrieve the data container, only if it's populated.
 29 
 30         """
 31 
 32         if  self.content:
 33             return self.content
 34         else:
 35             print "Error. Data not retrieved properly"
 36 
 37 
 38 class from_url_retriever (data_retriever):
 39     """
 40     This class is a child of data_retriever. Inherits from it the variable
 41     content, which serves as container for all the data, and the method
 42     return_data, which serves for returning this data container.
 43 
 44     This class adds the functionality to retrieve data from an url, using
 45     urllib2 module, and defines the methods retrieve_from_url, which returns
 46     an open file contining the content of the Url, and print_content, which
 47     prints the whole content of the given Url.
 48 
 49     """
 50 
 51 
 52     def __init__ (self, url):
 53         data_retriever.__init__ (self)
 54         self.url = url
 55 
 56 
 57     def retrieve_from_url (self, url):
 58 
 59         """
 60         This method uses the function urlopen from urllib2 module and
 61         retrieves all content of the site to url_content, which is
 62         a kind of file-object, already open
 63 
 64         This method handles the URLError and ValueError exceptions, and exits
 65         with an error message
 66 
 67         """
 68 
 69         try:
 70             url_content = urllib2.urlopen(url)
 71 
 72         except urllib2.URLError:
 73            error_message =  "Your url <" + url +  ">  is wrong, please fix"
 74            sys.exit(error_message)
 75 
 76         except ValueError:
 77             error_message = "Your url <" + url +  "> has an unknown url" \
 78                             " type. Please fix"
 79             sys.exit(error_message)
 80 
 81         else:
 82             return url_content
 83 
 84 
 85     def store_content (self):
 86 
 87         """
 88         This method calls for retrieve_from_url method and stores its return
 89         in content.
 90 
 91         """
 92 
 93         self.open_file_content = self.retrieve_from_url (self.url)
 94         self.content = self.open_file_content.read()
 95         self.open_file_content.close() # The return of urllib2.urlopen() is a
 96                                        # kind of file, closing it for safety
 97 
 98 
 99     def print_content (self):
100 
101         """
102         Calls for store_content() and prints content if not empty.
103 
104         """
105 
106         self.store_content()
107         if self.content:
108             print self.content
109         else:
110             error_message = "Something went wrong. Fix it"
111             sys.exit(error_message)
112 
113 
114 class yahoo_share_retriever (from_url_retriever):
115 
116     """
117     This class is a child of from_url_retriever, which sets the link
118     for yahoo share value download service, and asks for it's argument
119     only the company's symbol. Instead of asking for the url as
120     from_url_retriever, it uses the download.finance.com service of Yahoo.
121 
122     In order to choose which company you want to search, we hardcode the
123     link splitted in 2 halves, and  we keep it in a list. once an instance
124     is created, the constructor asks for nasdaq_symbol, which we insert in the
125     middle of the 2 halves (position 1 in url list) and then we join the list
126     into one single string,which we then pass to from_url_retriever constructor.
127 
128     Using this class, everything is prepared for be able to print just by
129     creating an instance and calling print_content(). print_content overrides
130     print_content from the parent class, to strip the content which is provided
131     with '\r\n', and print an aproppiate message
132 
133     """
134 
135     def __init__ (self, nasdaq_symbol):
136 
137         if nasdaq_symbol.isalpha(): # Risky, dont' know if they are only
138                                     # only alphabets, but anyway...
139 
140             url = ["http://download.finance.yahoo.com"
141                            "/d/quotes.csv?s=", "&f=l1"]
142             url.insert(1, nasdaq_symbol)
143             url = "".join(url)
144 
145         else:
146             error_message = nasdaq_symbol + " is not a correct NASDAQ symbol"
147             sys.exit(error_message)
148 
149         from_url_retriever.__init__ (self, url)
150         self.nasdaq_symbol = nasdaq_symbol.upper() # We keep it for output
151 
152 
153     def print_content(self):
154 
155         """
156         Overriden method that strips content to adapt the concret content of
157         the Yahoo service, and prints an aproppiate message.
158 
159         Yahoo service will generate a cvs file even if the symbol is incorrect.
160         It will contain 0.00, which is not good. This code makes sure it's
161         a desired output, or tells the user it's a wrong symbol
162 
163         """
164 
165         self.store_content()
166         self.content = self.content.rstrip("\r\n")
167 
168         if self.content.find("0.00") >= 0:
169             error_message = "The NASDAQ symbol '" + self.nasdaq_symbol +"' is" \
170                             " incorrect. Please make sure you provide a" \
171                             " correct symbol."
172             sys.exit(error_message)
173 
174         else:
175             print "Share value for %s: %s" % (self.nasdaq_symbol, self.content)
176 
177 
178 if __name__ == "__main__":
179 
180     if len(sys.argv) == 2:
181         yahoo_share_retriever (sys.argv[1]).print_content()
182         sys.exit(0)
183 
184     else:
185         sys.exit("Usage: ./sharevalue.py <NASDAQ_SYMBOL>")

sharevalue.py

#!/usr/bin/env python
"""
This python script will use the API modules
availables to check the value of a specific
ticker online and return it
"""

import sys
from YahooAPI import YFinance


if __name__ == '__main__':
    """
    We start by testing the number of arguments
    and exit if the number of arguments is less
    than 2
    """

    if len(sys.argv) != 2:
        print """
        Incorrect argument length

        Usage:
        sharevalue.py <ticker>
        """
        sys.exit(1)

    """
    We create a class from the Yahoo API
    We set the ticker argv[1]
    We ask for the value of the market
    """
    finance = YFinance()
    finance.link = sys.argv[1]
    value = finance.value

    """
    We test if value is set first
    if it is not we exit because the ticker is wrong
    We, then, test if value is set to -1
    if it is we exit because we couldn't connect
    """
    if not value:
        print """
        You submitted a ticker that doesn't exist
        Please try a different ticker than %s
        """ % (sys.argv[1])
        sys.exit(1)
    if value == -1:
        print """
        We were able to connect to the website
        The website might be down
        Please check your connection and try again.
        """
        sys.exit(1)

    """
    If everything goes well, we print the value
    """
    print """
        The current value of %s is %.2f
    """ % (sys.argv[1], value)

YahooAPI.py

"""
Yahoo API Module
================

YahooFinance Class
------------------
This class will save the ticker in a correct link format
and will output the correct value when value is called

"""

import urllib2


class YFinance(object):

    """
    The YahooAPI has a correct link
    and the value returned from Yahoo Finance
    """

    def __init__(self):
        """
        The default link is None
        """
        self._link = None

    @property
    def link(self):
        """
        This will return the link when needed
        to access the website and get the value
        """
        return self._link

    @property
    def value(self):
        """
        We try to connect to the link
        if everything goes well, we get the value
        we check if the value is not 0 and return it
        if we can't connect we will return -1
        """
        try:
            fobj = urllib2.urlopen(self._link)
            data = fobj.read()
            value = float(data)
            fobj.close()
            if value == 0.0:
                return
            return value

        except IOError:
            return -1

    @link.setter
    def link(self, ticker):
        """
        The link recieves a ticker and saves it as a link
        The link will be used later to get the information
        from the web page.
        """
        link = "http://download.finance.yahoo.com/d/quotes.csv?s=%s&f=l1" \
            % ticker
        self._link = link