Let us begin! [19:00] ----------- Packaging RPM session ----------- Roll call! Please tell your name! *** ANKUR_ (n=ANKUR@124.124.233.29) has joined channel #dgplug Ratnadeep Debnath Harsh verma Meejanur Rahaman Chandana Boral zer0c00l, Arun SAG [19:01] Kishan Goyal ok, todays' presentation is about packaging RPM; [19:02] *** s111 (i=3b601f16@gateway/web/freenode/x-3fcb07004b9284b0) has joined channel #dgplug if you would like to talk, please type '!'; otherwise, don't interrupt the session; *** ANKUR_ (n=ANKUR@124.124.233.29) is now known as DEVIL_HEART *** kopecks (n=koyel@117.201.96.209) has joined channel #dgplug if you haven't downloaded the presentation, please download it from: http://shakthimaan.com/downloads/glv/presentations/packaging-red-hot-paneer-butter-masala.pdf [19:03] *** DEVIL_HEART (n=ANKUR@124.124.233.29) has quit: Client Quit *** DEVIL_HEART (n=ANKUR@124.124.233.29) has joined channel #dgplug Packaging software is very important for easier installation, and upgradation at customer deployments; it is also useful to keep track of dependencies that your package needs; so this presentation is on packaging Red Hat Package Manager, that is useful on Fedora distribution; [19:04] *** aunkit (n=aunkit@117.197.60.209) has joined channel #dgplug some of you may not like development, but, might be interested in automation; packaging is one of those tasks that you could look at; this presentation has been done to make a pun on the word RPM; [19:05] --> #n, means slide number 'n' *** s111 (i=3b601f16@gateway/web/freenode/x-3fcb07004b9284b0) has quit: Ping timeout: 180 seconds --> #1 you are welcome to redistribute, share this presentation with your friends; [19:06] --> #2 to work on packaging software, you will need to install fedora-packager; *** s111 (i=3b5c6a59@gateway/web/freenode/x-98bec72b8e4d59da) has joined channel #dgplug [19:07] it will obtain all the tools required for the same; --> #3 Only once, you should rpmdev-setuptree, which will create a list of directories; --> #4 --> #5 In general, you use your HOME directory to run rpmdev-setuptree; it creates a rpmbuild directory [19:08] and BUILD, BUILDROOT, RPMS, SOURCES, SPECS, SRPMS directories within it *** SkyRocknRoll (n=SkyRockn@122.164.57.64) has joined channel #dgplug *** s111_ (i=3b5c6a59@gateway/web/freenode/x-77304a05eb53b46a) has joined channel #dgplug projects have their own development cycle, and releases; roll call over ? *** sumitc (n=chatzill@unaffiliated/sumitc) has joined channel #dgplug [19:09] SkyRocknRoll: yes; it is 1910 IST SkyRocknRoll: if you want to say something, type '!' first; please don't interrupt the session, otherwise; *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug mbuf, sorry ok what distro packagers do is take a snapshot from the project release (upstream) and package it for their distribution *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 54 (Connection reset by peer) [19:10] *** s111 (i=3b5c6a59@gateway/web/freenode/x-98bec72b8e4d59da) has quit: Ping timeout: 180 seconds *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug so packaging is basically referred as 'downstream' a distro is a collection of packages from various software project teams, put in a media (DVD, for example) and given to you for use; subsequent upstream releases are again fetched by the packagers, and next package releases are done; a freeze is done to release a particular distro version; hence we have Fedora 9, 10, and now 11; [19:11] --> #6 Here I am assuming we have Paneer Butter Masala as upstream that someone has already prepared; *** djthequest (i=daf85072@gateway/web/freenode/x-fe645077a6e3ab36) has joined channel #dgplug When you take it, you always check the README and/or INSTALL file in it; [19:12] that is why in software development, we insist on writing one; here we see a just a simple .c file that is part of paneer butter masala; --> #7 Upstream 'cooks' have prepared a Makefile to build it; [19:13] It is a simple Makefile to compile the .c file; installation is just moving the executable to a PATH that the system knows to search for executables, usually /bin, /sbin, /usr/bin or /usr/sbin; clean, just removes the paneer-butter-masala --> #8 [19:14] so this upstream paneer butter masala should be packaged so Fedora users can use it; really? yes We like food! so one needs to write a .spec file that describes on how to package the same! we will go through some basic, required fields in a .spec file; [19:15] using the tools installed in fedora-packager, we will package it; Each .spec file should have a 'Name' that states what the package name is; --> #9 Version denotes the upstream version number [19:16] --> #10 Release denotes your .spec file release number --> #11 Summary defines a brief one line summary of the package --> #12 Group defines the possible group to which the package belongs to; There is a small collection of groups used in Fedora; [19:17] *** ravijha (n=rkjpro@125.20.11.34) has joined channel #dgplug *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 54 (Connection reset by peer) they are listed in /usr/share/doc/rpm-*/GROUPS your package should fit in either one of these groups; *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug --> #13 License specifies the license the upstream source is released in; [19:18] must be a F/OSS license; --> #14 URL should point to the upstream project web page; --> #15 Source0 should point to the upstream URL for the exact upstream snapshot release that you are taking to package it; [19:19] usually upstream package repositories are hosted in sourceforge.net, or freshmeat.net, or github.com --> #16 BUILDROOT defines as to where the package will be installed by the rpm tools; *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 104 (Connection reset by peer) *** kishan_ (n=kishan@117.99.166.124) has joined channel #dgplug [19:20] this is usually in rpmbuild/BUILDROOT/ --> #17 You can write comments in the .spec file, and all comments begin with a # --> #18 BuildRequires specifies what software, tools or dependencies are required to build this package; [19:21] there are some basic ones that are assumed to be already available, like gcc, make et. al.; *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug the wiki link at the bottom of the slide lists these exceptions; *** root (n=weechat@59.93.206.179) has joined channel #dgplug mbuf, ! so you don't need to specify gcc, make et. al.; [19:22] *** root (n=weechat@59.93.206.179) is now known as Guest27388 but, if you need to specify, separate them with a space as shown; rtnpro: shoot! mbuf, page 17, didn't understand %{ tmppath}/%{name}-%{version}- %{release}-root-%(%{ id u} -n) *** sam111 (n=sam111@59.96.31.22) has quit: Read error: 110 (Connection timed out) [19:23] *** ravijha (n=rkjpro@125.20.11.34) has quit: "Leaving" *** Guest27388 (n=weechat@59.93.206.179) has quit: Client Quit mbuf, EOF rtnpro: these are called macros, and I will come to that later; these macros are substituted for values mbuf, ok rtnpro: usually %{tmppath} corresponds to /home/foo/rpmbuild/BUILDROOT [19:24] *** s111_ (i=3b5c6a59@gateway/web/freenode/x-77304a05eb53b46a) has quit: Ping timeout: 180 seconds mbuf, ok rtnpro: and the install directory will with name, version macros that have been defined in the spec rtnpro: Name in the .spec refers to %{name} here [19:25] rtnpro: Version in the .spec refers to %{version} here rtnpro: when you install the rpm with the rpm tools, you will see how this directory is 'named' --> #19 *** test (n=test@59.93.206.179) has joined channel #dgplug [19:26] *** test (n=test@59.93.206.179) is now known as Guest63620 Requires specifies any packages or tools that are required for the package to be used in run-time for paneer butter masala to be used, we need 'Naan' (for example) one can also specify version numbers next to BuildRequires and Requires, but, that is generally not encouraged [19:27] unless there is a real reason to put exact dependency version numbers, it is best to avoid them; so upgrade from one distro release to another is smooth; --> #20 %description is a multi-line description of the package; [19:28] --> #21 the %prep section is usually for preparing the sources; --> #22 along with %setup, they are usually used for extracting the upstream sources; [19:29] after these you can also add any patches that are needed (we will look at patches, later) --> #23 %build tells how to build the sources; --> #24 in our case, we have one Makefile and one .c file; so just issue make will build the paneer-butter-masala for us; [19:30] --> #25 %install tells us where to install the build executables (or libraries) --> #26 before installing, we first remove the previous installation instance; $RPM_BUILD_ROOT and %{rpm_build_root} refer to the same location; [19:31] --> #27 since the Makefile has an 'install' target we just call 'make install DESTDIR...' to do the installation to the $RPM_BUILD_ROOT/sbin directory; [19:32] all commands are executed in sequential order below the defined %install and other tags; --> #28 %clean is should remove previous installs; --> #29 --> #30 [19:33] The %files section tells the packager as to what files are to be included in the packaged; In our case, we have only one executable, 'paneer-butter-masala' *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 104 (Connection reset by peer) --> #31 %defattr defines the default attributes that the installed files should have; [19:34] *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug so in this example, the files owner and group should be 'root' --> #32 *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 54 (Connection reset by peer) the executable is located in the sbin/ directory; %{_sbindir} is a macro for the same [19:35] --> #33 upstream sources should ship documentation or atleast a README file; *** Shrink (n=sgupta@redhat/shrink) has quit: Read error: 113 (No route to host) *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug so we say we are to put them in the %doc directory, which is usually /usr/share/doc in *nix systems; --> #34 Finally we have a %changelog entry for the .spec file; this lists the changes made to the .spec file; [19:36] --> #35 Each new entry starts with a '*', followed by week day in three characters; then month in three characters, day (should be 01) actually; followed by your name, e-mail address and finally in a single line [19:37] our upstream source snapshot is 2.0; this is our first .spec file, so it is 1 --> #36 Starting with -, we tell what change has been made; this is a very simple .spec file; [19:38] --> #37 *** aunkit (n=aunkit@117.197.60.209) has left channel #dgplug Now we shall see how the rpm tools are used on this .spec file to clean, build, install the package; --> #38 you can use the --clean option to rpmbuild on the .spec file as shown to clean the package; --> #39 --> #40 [19:39] --> #41 By default, before you build, rpm will invoke %clean; so you don't have to explicitly do it before building the package; --> #42 --> #43 as we progress, this summary table will list the options for keeping track of what we have learnt; --> #44 [19:40] we need to prepare the sources, as defined in %prep and %setup --> #45 we use -bp option to rpmbuild on the .spec file; --> #46 --> #47 it cleans first, as mentioned earlier; *** sankarshan (n=sankarsh@fedora/sankarshan) has quit: "Are you sure you want to quit this channel (Cancel/Ok) ?" [19:41] and it extracts the upstream sources (.tar.gz) using tar; *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 104 (Connection reset by peer) you will need to manually keep a copy of the upstream sources in the rpmbuild/SOURCES directory; the .spec files are to be kept in the rpmbuild/SPECS directory whatever you build is in the rpmbuild/BUILD directory *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug whatever you install will be in the rpmbuild/BUILDROOT directory; whatever RPMs that are generated will be in the rpmbuild/RPMS directory; [19:42] --> #48 *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 104 (Connection reset by peer) so, we have extracted the sources; --> #49 then we build; --> #50 *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug *** kishan (n=kishan@117.99.167.61) has quit: Connection timed out [19:43] -bc to rpmbuild on the .spec file compiles the sources, or does make as mentioned in the .spec file; --> #51 --> #52 *** kishan_ (n=kishan@117.99.166.124) is now known as kishan --clean called by default; --> #53 --> #54 make is invoked on the extracted Makefile from the upstream sources; [19:44] *** zer0c00l_ (n=zer0c00l@117.199.140.52) has joined channel #dgplug --> #55 so the executable is built; at any stage if things are incorrect, rpmbuild will exit with error; the reason why I pasted the log in the slides, is for you to see what rpmbuild says; it is always good to read the logs! --> #56 --> #57 [19:45] now we need to install the executable in rpmbuild/BUILDROOT; --> #58 --> #59 --clean invoked first, by default; --> #60 --> #61 make is invoked to build it; [19:46] --> #62 --> #63 it installs the executable in DESTDIR; *** vivek (n=vivek@117.201.97.14) has joined channel #dgplug so what you see as paneer-butter-masala-2.0-1.i386 is the %{name}-%{version}-%{release} that you saw in the .spec file BUILDROOT [19:47] --> #64 installed! note /usr/sbin within BUILDROOT/packagename-version-release when you install the package on your system, using yum it will just install in /usr/sbin; --> #65 --> #66 [19:48] we already add the README file to be put in the %doc directory --> #67 --> #68 DOCDIR is defined and exported by default by the rpm tools; --> #69 README file was added to %files, so it is copied to the /usr/share/doc/paneer-butter-masala-2.0 directory [19:49] *** hermes1 (n=abhishek@114.143.50.205) has joined channel #dgplug --> #70 summary on what we have done so far; *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 104 (Connection reset by peer) *** vivek (n=vivek@117.201.97.14) has quit: Client Quit --> #71 now we package it as an .rpm; --> #71 *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug --> #72 -bb is used to create the .rpm; [19:50] --> #73 --> #74 --clean called by default and sources are extracted; --> #75 --> #76 build is done with make; --> #77 --> #78 installation is done at DESTDIR; --> #79 --> #80 [19:51] --> #81 %doc needs to be processed; --> #82 --> #83 DOCDIR defined and exported; --> #84 README is copied to the respective directory for the package; --> #85 --> #86 [19:52] *** zer0c00l_ (n=zer0c00l@117.199.140.52) has quit: "will check logs" a binary .rpm is created! *** sumit (n=chatzill@125.20.11.34) has joined channel #dgplug --> #87 --> #88 *** sumit (n=chatzill@125.20.11.34) has quit: Client Quit so if you check the rpmbuild/RPMS/i386 directory, you will see the .rpm generated; the debuginfo rpm is has debug symbols in it, so if a user has a problem with the package, they can install this package and get a core dump, that can be sent to the developer for bug fixing or troubleshooting; [19:53] --> #89 so there you have the summary for creating an .rpm with rpmbuild tool; --> #90 The binary rpm only has the executable; but if you want to make a package of the sources, you can make a .src.rpm; [19:54] you use -bs with rpmbuild on the .spec file to do it; --> #91 I have skipped the logs here, and just showing the name of the built rpm; --> #92 summary, again; --> #93 if you want to build both .src.rpm and .rpm you can also use -ba with rpmbuild; [19:55] note the SRPMS are located in rpmbuild/SRPMS --> #94 --> #95 --> #96 --> #97 that is the complete summary of what we have done so far; --> #98 rpmlint is useful to do some sanity checks on the .spec file that you have written; [19:56] just to make sure that your .spec file follows the standards defined for RPM; --> #99 *** hermes1 (n=abhishek@114.143.50.205) has left channel #dgplug if there are no errors or warnings, you will get an output like this; --> #100 [19:57] you should also run rpmlint on the built .rpm to check for errors or warnings; --> #101 --> #102 also on the .src.rpm --> #103 --> #104 *** zer0c00l (n=zer0c00l@117.199.131.143) has quit: Connection timed out *** hermes1 (n=abhishek@114.143.50.205) has joined channel #dgplug If you simply want to list the rpm contents, you can use rpmls on the .rpm; --> #105 as you can see it lists the executable, paneer-butter-masala to be installed in /usr/sbin and the README file [19:58] --> #106 you can also use rpm2cpio, cpio to extract the RPM contents [19:59] --> #107 --> #108 so, we can now test the rpm; you can use yum install with --nogpgcheck to install it; if the RPM is signed with your GPG check and added to yums' list, then nogpgcheck is not required; --> #109 [20:00] it will prompt you if you want to install it or not, and if you say 'Y' it will install it; --> #110 you can check if the rpm is installed or not through -qa option to rpm; --> #111 --> #112 since the executable is in the PATH, you can simply run to see the output; --> #113 [20:01] that is what the upstream .c file printed, anyway; --> #114 if you want to uninstall the rpm you can use 'yum remove' and just the package name! note that we haven't specified any version, release or i386 (arch) type; --> #115 now we are going to go through the process of how to create patches; [20:02] *** sumitc (n=chatzill@unaffiliated/sumitc) has quit: Read error: 110 (Connection timed out) so you have taken a software from upstream and you found a bug in it, or you feel something needs to be changed; you can make the change as a patch; add the patch to the .spec file so when you package it, your changes are in the built package that you ship with your distro; you can send these changes to the upstream source; [20:03] the upstream project developers may or may not take it; I am going to describe one way of creating a patch; use -bp to extract the sources; --> #116 --> #117 --clean called and the sources are extracted; --> #118 you know where the sources are extracted, as in rpmbuild/BUILD/packagename [20:04] --> #119 enter into the directory; since we have only one .c file in our upstream source, and we want to make a change in it, we make a copy first with a .fix extension --> #120 I am now making changes in the original file; --> #121 go one directory up -- to the rpmbuild/BUILD directory; [20:05] --> #122 use gendiff on the packagename and the extension that we used (.fix) --> #123 this basically lists the change as a diff output; *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 104 (Connection reset by peer) as you can see, I only changed the printf; - is the old, + is the new change; --> #124 [20:06] --> #125 *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug to save it as a patch, I just redirect the output as a paneer-fix.patch file this .patch and all patches must be in the rpmbuild/SOURCES directory; --> #126 now we are going to make a new release of the package, with an updated .spec file [20:07] our Version (upstream) is still 2.0; our Release for the .spec file is now 2; --> #127 We give the name of the patch and it is referred as Patch0; we can have many patches like Patch1, Patch2, etc. [20:08] --> #128 we need to apply the patch on the sources, as soon after they are extracted; so it is after the %prep, %setup phase; we say %patch0 has to be applied --> #129 that is for the .c file extension [20:09] --> #130 We added a new changelog entry, the recent one on the top; *** yevlempy (i=cbc8bc02@gateway/web/freenode/x-30d7fc39fa830d4a) has quit: "Page closed" *** akuscifi007 (n=akuscifi@122.162.83.132) has quit: Read error: 104 (Connection reset by peer) so the updated date was on May 2, compared to the previous May 1 [20:10] *** chandana (i=cbc8bc02@gateway/web/freenode/x-a8a4c023911a6aa6) has quit: "Page closed" Also see that we have incremented the release from 2.0-1 to 2.0-2 *** akuscifi007 (n=akuscifi@122.162.83.132) has joined channel #dgplug a new comment that we have 'Fixed Paneer Butter Masala' has been added --> #131 --> #132 we can simply test if the patch applies cleanly; just -bp will do the %prep, %setup and called %patch0 [20:11] --> #133 --> #134 clean invoked and the sources are extracted; --> #135 patch is applied and the exit status is 0 -- which is clean; [20:12] --> #136 here you find a short list of macros that are used in the .spec file; and what each one corresponds to in the file system; when packaging, one should always follow Filesystem Hierarchy Standard (FHS) as to where one should put the files in the / directory; --> #137 you can define your own macros if you want to; [20:13] you use the %define tag to do it; here I show an example of redefining the destdir target; --> #138 --> #139 --> #140 sometimes the upstream software might use a specific language, say python, or perl, or ruby [20:14] and the distro defines that such packages need to named as python-foo, or perl-foo, or ruby-foo; so this example defines a macro packname and how to use the same in 'Name' --> #141 the wiki page has useful content on RPM macros; --> #142 [20:15] some quick notes/tips if you like; it is good to use a separate username for building and testing packages rather than your regular user account that you use in your system just to check on permission issues (if any), dependency package access and so on; [20:16] --> #143 if you have a macro within a comment (#) in the .spec file, the rpmbuild tool might actually parse the macro; so avoid this; --> #144 Koji is used (online) to build your RPM across different architectures to make sure it builds fine; [20:17] if your upstream package uses interpreted language, like perl, ruby, python then your package should use 'Arch: noarch' in the .spec file; so when built, it will be built in rpmbuild/RPMS/noarch as compard to rpmbuild/RPMS/i386 that we have seen here; [20:18] --> #145 --> #146 --> #147 --> #148 --> #149 --> #150 the documentation in fedoraproject.org wiki is quite good; please go through it when you get the time; [20:19] --> #151 --> #152 --> #153 --> #154 --> #155 --> #156 --> #157 there are useful mailing lists, as well as #fedora-devel for your packaging queries; you can also ask in #fedora-india what I have described is just how to package RPMs; but there is a workflow in how to get your built packages approved by Fedora sponsors; [20:20] for that, I have written this draft document: http://shakthimaan.com/downloads/glv/howtos/packaging-rpm-workflow.html please go through it; the presentation + packaging rpm workflow should help you doing your first RPM package (I hope) If you like this work, I would appreciate your help in packaging software for Fedora; [20:21] we also have packaging tasks in Fedora Electronic Lab, if you are interested; https://fedorahosted.org/fedora-electronic-lab/report/1 [20:22] you could take one task at a time; with this, the presentation ends; I shall take any questions that you might have;