Commit 4badb2e4 authored by Gerson Sunyé's avatar Gerson Sunyé
Browse files

continuous integration, maven

parent 7f0aecb9
......@@ -120,6 +120,22 @@
<section data-markdown="slides/" data-separator="^\n---$" data-separator-vertical="^\n----$"></section>
<section data-markdown="slides/" data-separator="^\n---$" data-separator-vertical="^\n----$"></section>
Continuous Integration
Improving Software Quality And Reducing Risk
Build Software Every Change
Build may consist of the compilation, testing, inspection, and deployment.
CI scenario typically go like
Developer commits code to version control repository
CI server detects that changes have occurred in the version control repository, and then executes a build script
CI server generate feedback by e-mailing build results to specified project member
CI server continues to poll for changes in the version control repository
Features of CI
Source Code Compilation
Database Integration
Document and Feedback
How could you know you are doing CI correctly?
Are you using a version control repository (or SCM tool)?
Is your project’s build process automated and repeatable? Does it occur entirely without intervention?
Are you writing and running automated test?
Is the execution of your tests a part of your build process?
How do you enforce coding and design standards?
Which of your feedback mechanisms are automated?
Are you using a separate integration machine to build software?
What prevents teams from Using CI?
Increased overhead in maintain the CI system
Too much change
Too many failed builds
Additional hardware/software cost
Developers should be performing these activities
How Do I Get to “Continuous” Integration?
Identify – Identify a process that requires automation
Build – Creating a build script makes the automation repeatable and consistent
Share – By using a version control system
Continuous – Ensure that the automated process is run with every change applied
Is it Continuous Compilation or Continuous Integration?
How much code coverage do you have with your test?
How long does it take to run your builds?
What is your average code complexity?
How much code duplication do you have?
Are you labeling your builds in your version control repository?
Where do you store your deployed software?
How does CI complement other development practices?
Developer testing
Coding standard adherence
Small release
Collective ownership
CI and you
Commit code frequently
Don’t commit broken code
Fix broken builds immediately
Write automated developer tests
All tests and inspections must pass
Run private builds
Avoid getting broken code
Risk: Lack of Deployable Software
Scenario: “It works on My Machine”
Solution: Use a CI server along with an automated build using tools such as Ant, NAnt, or Rake
Scenario: “Synching with the Database”
Solution: Place all database artifacts in your version control repository
Scenario: “The Missing Click”
Solution: Using script to automate the deployment process
Risk: Late Discovery of Defects
Scenario: Regression Testing
Solution: Using unit test at the business, data, and common layers and run continuously as a port of your CI system
Scenario: Test Coverage
Solution: Running test coverage tool to assess the amount of source code that is actually executed by the tests
Risk: Lack of Project Visibility
Scenario: Did you Get the Memo
Solution: Automated mechanism that sends e-mails to affected parties when a build fails
Scenario: Inability to Visualize software
Solution: Automated code documentation tool
Risk: Low-Quality Software
Scenario: Coding standard adherence
Solution: Using Checkstyle and PMD to report any lines of code that were not meeting the established standards
Scenario: Architectural Adherence
Solution: Using analysis tools such as JDepend or NDepend
Scenario: Duplicate Code
Solution: Automated inspection tools such as PMD’s CPD or the Simian static analysis tools
Integration build Scalability and Performance
Gather build metric
Analyze build metric
Choose and perform improvements
Reevaluate; repeat if necessary
Continues Database Integration
Automate Database Integration
Use a local database sandbox
Use a version control repository to share database asset
Give developer the capacity to modify database
Make the DBA part of the development team
Reduce Code Complexity
Cyclomatic Complexity Number (CCN) is a plain integer that measure complexity by coding the number of distinct paths through a method
Various studies with this metric over the years have determined that methods with CCN greater than 10 have a higher risk of defects
The most effective way to reduce cyclomatic complexity is to apply the extract method technique
Perform Design Reviews Continuously
Afferent Coupling, Fan In, an object has responsibility to too many other objects (high afferent)
Efferent Coupling, Fan Out, the object isn’t sufficiently independent of other objects (high efferent)
Instability = Efferent Coupling/(Efferent Coupling + Afferent Coupling)
Maintain Organization Standards with Code Audits
Coding standard facilitate a common understanding of a code base among a diverse group of developer
Human code reviews and pair programming can be effective in monitoring coding standards, they do not scale as well as automated tools
Popular code analysis tool for Java platform is PMD
Reduce Duplicate Code using PMD-CPD or Simian
CI Resources
Automated Inspection Resources
CI Resources
Deployment Resources
FeedBack Resources
Ambient Devices
Google Talk
Documentation Resources
Evaluating tools
Compatibility with your environment
Does the tools support your current build configuration?
Does the tool require installation of additional software in order to run?
Is the tool written in the same language as your project?
# Continuous Integration
## Plan
- Introduction<!-- .element: class="fragment highlight-blue" -->
- Control Version Systems
- Test Automatization
- Build Automation
- Feedback Mechanisms
- Continuous Deployment
- Conclusion
## Introduction
## Continuous Integration
« Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day.
Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.» [Fowler]
## In other words...
- Software integration is often perceived as a complex and annoying task.
- Continuous integration simplifies this task, making it part of developers daily work.
## A simple technique
- A simple yet powerful technique allowing software development teams to:
- keep team members in harmony.
- reduce potential drift hazard.
- limit complexity
- Regular build and test of the latest version of the software under development.
- Local testing and propagation to a single storage location of each developer's work.
## Motivation (1/3)
- Time saving:
- no more repetitive tasks.
- reduced need for documentation.
- better integration of newcomers, which can analyze and use build scripts directly.
- Developers' confidence improvement:
- Independent from the momentary/punctual efficiency of the developers.
- Use of reproducible procedures.
## Motivation (2/3)
- Risk reduction:
- errors are detected and fixed very early.
- software quality this measurable over time.
- reduction of assumptions.
- Manual task reduction:
- the build process is invariable: it follows the same sequence of tasks.
- the process runs automatically after each change/commit.
## Motivation (3/3)
- Availability of ready-to-use software at any time and place.
- Project visibility improvement:
- effective decisions.
- observable evolutions.
- Establish greater confidence in the software product from the development team
One of the most significant problems in software development is assuming
CI help to mitigate
- Lack of cohesive, deployable software
- Lack defect discovery
- Low-quality software
- Lack of project visibility
This facilitates
The reduction of labor on repetitive process
The capability to overcome resistance to implement improvements by using auto mated mechanisms for important processes
## Continuous Integration Components
- Control Version System (Repository)
- CI Server
- Build Scripts
- Feedback Mechanisms
- Integration
- Build Machine
## Plan
- Introduction
- Control Version Systems<!-- .element: class="fragment highlight-blue" -->
- Test Automatization
- Build Automation
- Feedback Mechanisms
- Continuous Deployment
- Conclusion
## Control Version System
- Unique place of sharing.
- Regression, snapshots, branches, etc.
- Tools: Git, Mercurial, Subversion, etc.
## Plan
- Introduction
- Control Version Systems
- Test Automatization<!-- .element: class="fragment highlight-blue" -->
- Build Automation
- Feedback Mechanisms
- Continuous Deployment
- Conclusion
## Test Automatization
- Developers:
- participate in the creation of the tests
- validate their code locally.
- Tools: JUnit, Selenium, SonarQube, PMD, FindBugs, etc.
## Unit Test Automatization
- Unit test verify the behavior of small elements in software system
- Mocks are simple object that substitute for real, more complicated object
- The key aspect for unit test is having no reliance on outside dependecies
## Integration Test Automatization
- Component or subsystem test verify portions of a system and may require a full installed system or same external dependencies
- Component level test use more dependencies than unit test but still not necessarily as many as high level system tests
## System Test Automatization
- System test exercise a complete software system and therefore require a full installed system
- System test are fundamentally different form functional test, system test much like a client would use the system
- Automate Functional tests
- Functional tests test the functionality of an application from the view point of a client, known as acceptance tests
## Continuous Test Evaluation
- Assess Code Coverage:
- Line coverage, statement coverage, indicates that a particular line of code was exercised
- Branch coverage measure the coverage of decision points
- Test run slower than they do in no coverage scenarios; It may be appropriate to run code coverage tools as part of a secondary, more heavyweight build
## Continuous Inspection
- Code Review can be impressively effective; however, they are run by humans, who tend to be emotional
- Pair Programming has also been shown to be effective when applied correctly
- Automated static code analysis scales more efficiently than humans for large code bases
- What is the different between inspection and testing
- Testing is dynamic and executes the software in order to test the functionality
- Inspection analyze the code based on a set of predefined rules
- Example of inspection targets include coding “grammar” standards, architectural layering adherence, code duplication, and so on
## Plan
- Introduction
- Control Version Systems
- Test Automatization
- Build Automation<!-- .element: class="fragment highlight-blue" -->
- Feedback Mechanisms
- Continuous Deployment
- Conclusion
## Build Automatization
- Non-trivial Software is build using build management tools.
- The goal of a build management tool is to fully automate all steps required to build the product given the source artifacts of the project.
- The result of the build should always be the same - independently of the developer’s local configuration.
- Builds must be stable.
- The build management tool is responsible for automatically carrying out all steps necessary to build the product.
## Build Automatization Tools
- Client tools: scripts to compile, assemble, generate, deploy, etc.
- Schedulers: automation scripts in the server side (e.g. crontab).
- Examples: Ant, NAnt, Maven, Gradle, Rake, etc.
## Build Tools Functionalities
- A Build Automation typically executes the following tasks:
- Source Code Formatting
- Code generation
- Code compilation
- Component packaging
- Program execution
- File manipulation
- Extended taks:
- Development test execution
- Version control tool integration
- Generating and publishing documentation, release notes, web pages, etc.
- Deployment to the test system/production system(s)
- Code quality analysis
- Extensibility
- Multiplatform builds
- Accelerated builds
## Build Schedulers Functionalities
- Essential:
- Build execution
- Version control integration
- Build tool integration
- Feedback
- Build labeling
- Extended:
- Inter-project dependencies
- User interface
- Artifact publication
- Security
## Build Types and Mechanisms
- Types:
- Private Build
- Integration Build
- Commit build
- Secondary build
- Release Build
- Mechanisms:
- On demand (e.g., by a developer)
- Scheduled by a build server (e.g., every night)
- Poll for changes
- Event-driven (e.g., on every commit to a version control system)
## Plan
- Introduction
- Control Version Systems
- Test Automatization
- Build Automation
- Feedback Mechanisms<!-- .element: class="fragment highlight-blue" -->
- Continuous Deployment
- Conclusion
## Feedback Mechanisms
- Send the right information to the right people at the right time and in the right way:
- Email
- Windows taskbar monitor
- Sounds
- Collaboration tools: IRC, RSS, Slack, Mattermost, Fleep, etc.
## Typical Deployment
## Active Operation
- Developers propagate their changes.
- The integration server monitors the control version system.
## Continuous Integration Tools
- Hudson, Jenkins.
- CruiseControl / CruiseControl.NET.
- Apache Continuum.
- PMEase QuickBuild (open-source: LuntBuild).
- And several others AnthillPro, Bamboo, BuildForge, Continuous Integration Server Matrix, Draco.NET, Gauntlet, ParaBuild, etc.
## Plan
- Introduction
- Control Version Systems
- Test Automatization
- Build Automation
- Feedback Mechanisms
- Continuous Deployment<!-- .element: class="fragment highlight-blue" -->
- Conclusion
## Continuous Deployment
- Release working software any time, any place
- Label a repository’s assets
- Produce a clean environment
- Label each build
- Run all tests
- Create build feedback reports
- Possess capability to roll back release
## Plan
- Introduction
- Control Version Systems
- Test Automatization
- Build Automation
- Feedback Mechanisms
- Continuous Deployment
- Conclusion<!-- .element: class="fragment highlight-blue" -->
## Practices of Continuous Integration
### According to Martin Fowler
- Maintain a Single Source Repository.
- Automate the Build
- Make Your Build Self-Testing
- Everyone Commits To the Mainline Every Day
- Every Commit Should Build the Mainline on an Integration Machine
- Fix Broken Builds Immediately
- Keep the Build Fast
- Test in a Clone of the Production Environment
- Make it Easy for Anyone to Get the Latest Executable
- Everyone can see what's happening
- Automate Deployment
## Conclusion
- Continuous Integration essential aspects:
- Version control
- Automatic and fast build
- Code quality
- Information share
## References
- «Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation». Jez Humble and David Farley.
......@@ -8,5 +8,7 @@
# Apache Maven
## Foreword
«Maven was born of the practical desire to make several projects at the Apache Software Foundation(ASF) work in the same, predictable way.