issue_classify/background.tex

\subsection{Social Coding}
Open source projects aim to attract many external contributors, who can provide ideas, codes, issues and so on to develop the project.
These external contributors may come form different countries and use different language.~\cite{hertel2003motivation}.
Compared with traditional development approach, geographic and temporal differences are more common in open source.
Nowadays, open source communities supply many tools integrated with multiple social media features, which let users track and follow the activities of a large number of others regardless of location or affiliation~\cite{dabbish2012social}.
These tools with social media provide a convenient way for developers to communicate, promoting more users to contribute to open source project.

In the social coding platform (\eg BitBucker, GitHub), the transparency makes developers can discuss anything with anybody.
The frequent and convenient communication get better development efficiency and software quality.
For instance, Reporting issues using ITS is one of the most important activities in social coding communities.
Anyone in the platform can use the ITS to report bugs of software and discuss issues based on comments of ITS.
And the core team can utilize the ITS to assign the issue to related developers and manage development task.
The interactive process of managing issues makes developers get more involved and narrow the distance between the core team and external contributors.
This process becomes a win-win situation.
On the one hand, external contributors can participate in projects and gain reputation.
Developers can gain a sense of accomplishment in the process of reporting, discussing and fixing issues.
And providing contributions to a excellent projects can make developers acknowledge to others.
On the other hand, it is helpful for open source software to acquire more contributions.
The Converity scan report~\cite{converty2014} shows the quality of open source software (defect density of 0.61) is considerably better than commercial software (defect density of 0.76), which chooses 10,000 most active projects and analyzed more than 10 billion lines of code across C/C++, Java and C\# projects.
The report also shows that more attention and contributions improve the quality of open source software and reduce the time cost of fixing defects.
% social coding 的发展吸引了大量的人员参与其中，并为之提供贡献。其中提交缺陷报告是最重要的互动之一。
% 社交化变成降低了门槛，

\subsection{Issue Tracking System}
\label{ITS_T}

Software development generally produces programs with two caveats~\cite{bissyande2013got}:
(1) they are often incomplete with respect to certain features,
and (2) they are usually buggy.
In the development process, most of time, developers code and test the programs,
while end-users (does not exclude developers) use the programs and provide feedbacks.
Both developers and end-users can submit issues to ITS
when the software performance does not meet their expectations.
%And submitting issues to ITS is one of the most important contribution way for OSS participants~\cite{jalbert2008automated}.
%Good management of developing tasks is of great benefit to improve develop efficiency.
%During the process of issue management,
%the core team needs to make textual information of the issue clear,
Then, the core team needs to clearly understand the intentions of the contributors,
distinguish categories, and find suitable developers to fix the corresponding issues
at the stage of project maintenance.
% For core team of a project,
% they need to tell what kind of tasks they have and decide who should fix them.
% They are supposed to communicate with other developers and coordinate the work among them.
Using the ITS is a common way to organize and maintain
development tasks in the open source practice~\cite{zimmermann2009improving} 
to help project mangers keep track of issue reports by monitoring progress,
identifying new issues, discussing potential solutions for fixing bugs, and so on.
The consistent utilization of ITS is considered as a
``hallmarks of a good software team''~\cite{spolsky2010painless} in open source communities.
% Using ITS is of great benefit to the management of developing tasks.

\begin{figure}[!htb]
\centering
% \includegraphics[width=8.5cm]{classprocess}
\includegraphics[width=6cm]{figure/workflow_bugzilla}
\caption{Workflow of management for traditional ITS}
\label{figure:lifecycle_old}
\end{figure}%picture

Dozens of ITS tools has been popularized,
\eg \emph{Bugzilla} and \emph{ITracker}, 
with the development of OSS.
%In the development of ITS, many ITS appear, such as Bugzilla and ITracker~\cite{serrano2005bugzilla}.
% 添加传统管理工具的使用
These traditional tools design a rigid and complicated data structure,
which are used for organizing issues, \eg category, priority, assignee and status.
%These tools provide well organized task management with
%structured information (\eg category, priority, assignee, etc) and in this paper,
%ITS like these are called traditional ITS.
Figure~\ref{figure:lifecycle_old} shows the common workflow.
First, when contributors find bugs in the software, 
they are usually asked for a basic description about the bug, 
and the structured fields are completed to as many as they can. 
Second, core team members and contributors discuss their problems to clearly understand the problems. 
Thereafter, the structured information that requires distinguishing suitable categories, 
determining priorities, making plans to ensure progress, and locating the issue 
(indicating the product, component, and version of the software where the issue appeared) are corrected. 
Finally, based on all structured information, the corresponding developers would be assigned to fix the bugs.

%These ITSs provide many comprehensive but complex structured information compared with their predecessor,
%mailing-list and spreadsheet~\cite{serrano2005bugzilla}, and these structured information is great benefit to manage and retrieve issues.
%Through the long development cycle, projects have collected many informationally complete issue reports and modification history,
%and these issues contain many meaningful informations for managers and researchers.

% Through the whole life cycle, the issue tracking system collect many informationally complete issue reports and modification history of them, which contain many meaningful informations for managers and researchers.




%It also leads some research on the construction of improving ITS~\cite{just2008towards,zimmermann2009improving}.

% Developers use unstructured free text to describe issues and use structured information like category, priority, etc, to manage ITS.
% Through these structured information, manager can decide when and who should to fix them and record whether it appeared before.

% In the development of ITS, there are appearing many ITS such as Bugzilla and ITracker \cite{serrano2005bugzilla}.
% These ITSs have many comprehensive and complex features compared with their predecessor, mail-list and spreadsheet \cite{serrano2005bugzilla}.
% These features, such as category, priority, status, etc, are designed for recording developing activity as much as possible, and it works effectively compared with mail-list.
% However, some recently researches \cite{antoniol2008bug,herzig2013s,kim2011dealing,zhou2014combining} observed that a considerable amount of issue reports in ITS marked as defective actually never had a bug.
% Too many default values are selected for options of issue reports, which make most function of issues meaningless.
% Reports with wrong information bring trouble to managing projects, and also hamper the researches on ITS.
% The ITS data with correct information become the key to make a successful analysis.
% To make sure that developer have provided correct information, it cost much time and manpower for manager.
% And this situation seems to be more intense with the rapid growth of the number of issues.
% Manual classification can help reduce the misclassification but the rapid growth of the number of issues make it an impractical option.
% Therefore, automatically classifying issue report would be very useful.

% \subsection{Lightweight ITS in GitHub}
% \label{ITS_GH}
%GitHub is a social coding OSS community, which allows contributors of GitHub to communicate with each other by social media tools.
%Social media tools play an increasingly important role in software engineering research and practice~\cite{storey2010impact}.
%And the usage of social media drive a rich set of inferences around commitment, work quality,
%community significance and personal relevance~\cite{dabbish2012social},
%which helps GitHub become the largest platform of providing web-based repository hosting service.



GitHub, the largest social coding community, 
released its own ITS called \emph{Issues 2.0}\footnote{\url{https://github.com/blog/831-issues-2-0-the-next-generation}} 
in 2014 to provide an excellent service in reporting issues. 
Figure~\ref{figure:lifecycle} summarizes the typical workflow of issue management in GitHub.
First, the contributor submits an issue report and provides some textual summary to describe it.
Second, the core team of the project discusses the issue with the contributor.
During discussion, the core team needs to reach an agreement of the issue with the contributors, 
and select relevant labels for the issue from pre-defined labels. 
Finally, the core team assigns the issue to be fixed by the corresponding developer.

\begin{figure}[!htb]
\centering
% \includegraphics[width=8.5cm]{classprocess}
\includegraphics[width=6cm]{figure/workflow_git}
\caption{Workflow of management in GitHub ITS}
\label{figure:lifecycle}
\end{figure}%picture

GitHub provides a more lightweight ITS that is flexibly integrated with its label system, 
compared to traditional ITS. 
The structured information of issues, 
such as category and priority, 
is substituted by the label system in GitHub. 
Contributors are only required a short textual abstract when submit issues, 
whereas the core team can use labels, besides milestone and assignee, 
to mark and manage issue reports. 
The label system in GitHub is custom and the core team can summarize information concerns them most as labels.
% These predefining labels may contain the information of category, status, component, which is concerned most for the core team.
% there are only labels can be used to manage issue reports.
% It is hard to judge whether this change is benefit or not for projects,
Zach Holman, a GitHub engineer, describes his design as follows: \textit{``Our goal is basically to make a flexible, simple product that everyone can enjoy.''} \textit{``In the meantime, we've found that using labels and milestones is a great way to achieve the same result (other functions) in a more flexible system.''}
% The flexible ITS do not force users to provide structured information (e.g., category, priority, etc.) when users submit an issue report.
% Users just need to focus on what they want others know (title and description of issue report), and this flexible feature can reduce the fault of default values.

The lightweight design of ITS results in some changes for the contributors and core team. 
For contributors, Issue 2.0 reduced the cost of submitting issue reports and stimulated their enthusiasm. 
However, its openness results in the emergence of undesired issues in ITS. 
Consequently, loose constraints reduced the work of contributors, 
which are transferred to the core team as management task. 
Thus, an automatic approach that can effectively filter out useful issue reports is significant and urgent for issue management. 
For the core team, the flexible label system makes the management task customizable and configurable. 
The flexibility breaks the fixed form of management, 
and the core team can shape the way of management according to their requirement. 
However, this customization lacks mandate and enforceability, 
thereby omitting structured information. 
Moreover, this customization results in difference in usage among projects, 
which makes the management of different projects difficult to understand.

% But the core team \hl{is more likely to forget to add some labels} because of loose constraints,
% which aggravates the omitting of structured information.

% contributors easy to commit issue reports.
% Contributor do not need to provide much information, and transfer
% which correspondingly transfers the task to managers.
% In addition, the usability of this design let contributors commit issue reports more easily, while this results in committing information more casually than before.
% And this situation indirectly increased the management task.
% Many projects give up maintaining work of some information such as priority, component, because of too onerous management task.
% Missed information is also very important, but the limitation of manpower and time makes it impossible to keep information correct and accuracy.
% So import automatic technique to help management is great urgent.

% 这使得将一些自动化的技术引进变得更有意义。比如说自动分类
% 考虑到GitHUb与传统ITS的不同，我们提出了第一个RQ
% 机器学习的方法是否在ITS of GitHub适用，如果适用的话，哪种效果好

% 研究combine。。提出了使用一些结构化的信息来提升分类的准确率，但是对于GitHu来说，这些信息太珍贵。我们能够使用的信息只有文本以及提交者的历史活动信息，所以我们希望能够在文本中找到影响分类的一些因素。

% This flexibility, however, comes at some cost in managing the project.
% On the one hand, few users provide extension information which is important to managers of the project.
% Extensive loss of extension information cause more manpower needed to maintain ITS.
% So automatic issue reports classification is of great signification in GitHub.
% On the other hand, issue reporters may categorize the issues with typographical mistakes, or using various idiosyncrasies \cite{bissyande2013got}.
% Many labels are used to distinguish an issue report as a bug or feature, such as bug, defect, feature, feature request, etc.
% And it is hard to propose a unified method to distinguish these labels because of this complex situation.

%介绍issue system in github，并介绍相关工作

%\subsection{Relative Research in Automatically Classifying Issue Reports}
\subsection{Related Work}
\subsubsection{Social Coding}
For small groups, notifying members of actions on shared artifacts is common, which helps them maintain mental models of others activities~\cite{gross2005user} and avoid potential coordination conflicts~\cite{sarma2003palantir}.
However, for open source communities, there are often hundreds or thousands of individuals participating in developing projects online.
Transparency of others' actions in this type of setting is more different.
So project hosting platforms introduce social media to make users' behavior visible to others.
Developers can make inferences about others from what they observe and discuss what they are interested in.

With the popularity of social coding, developers can follow others' behavior and ignore location, and more and more developers join social coding platform and provide contribution.
Many works have studied social coding site and analysis social network in the platform.
For instance, some work focus on analyzing basic principle of social coding site. 
Laura et al.~\cite{dabbish2012social} studies the transparency in an open software repository and how they collaborate with each other in the repository.
Asher et al.~\cite{trockman2018adding} research the emerging phenomenon of repository badges about npm packages.
And Raphael et al.~\cite{pham2013creating} works on testing culture on social coding site.
They find several strategies that software developers and managers can use to positively influence the testing behavior in their projects by interviews and questionnaire.

In addition, there are some works on mining data in social coding platform.
Wang et al.~\cite{wan2018scsminer} developer a framework to locate the developers by mining coding skill from data in social coding site.
a number of studies analyze network structure of SourceForge~\cite{surian2010mining}, StackOverflow~\cite{treude2011programmers} and GitHub~\cite{thung2013network}.
These works try to mining social activities from the network structure and use the result to direct software development and recommend useful information for developers.

\subsubsection{Issue Tracker System}
Issue tracker is one of most important tools in open source communities.
It provides a platform for developers to discuss about finding and fixing bug, adding feature, managing development tasks, and so on.
And the core team spends large amounts of manpower to maintain the information in ITS.
Many academic studies focus on ITS to free the managers from some cumbersome and repetitive work,
%improve the efficiency of project maintenance,
%because of the key role that ITS plays in the development of project,
\eg for bug assignment task,
John et al.~\cite{anvik2006should} present a semi-automated approach intended to ease the assignment of reports to a developer based on text categorization machine learning technique.
Olga et al.~\cite{baysal2009bug} present a framework for automated assignment of bug-fixing tasks by employing preference elicitation to learn developer predilections in fixing bugs within a given project. 
Some researchers study the duplicate issue detection.
Wang et al.~\cite{wang2008approach} present an approach to detect duplicate bug reports by comparing natural language information and execution information of new reports with existing bug reports.
Sun et al.~\cite{sun2010discriminative} leverage recent advances on using discriminative models for information retrieval to detect duplicate bug reports more accurately.
More over, there are researchers working on predicting fixing time of issues.
Cathrin et al.~\cite{weiss2007long} present an approach that automatically predicts the fixing effort by using the Lucene framework to search for similar, earlier reports and using their average time as a prediction.


Most of these automatic tools and existing approaches for ITS highly depend on the structured bug data (\eg priority and severity).
%Otherwise, some transitional ITSs force contributors to apply much information to cut the workload of core team.
However, prior works~\cite{antoniol2008bug,herzig2013s,zhou2014combining}
have shown that OSS contributors often omit or use default value for some important information,
which results in many wrong messages and missing messages existing in ITS.
%which defeats the purpose of ITS.
Thus, improving the efficiency of ITS services is
becoming an important research topic~\cite{zimmermann2009improving}.


\subsubsection{Issue Reports Classification}
Many studies have investigated bug classification \cite{antoniol2008bug,herzig2013s,zhou2014combining} 
to predict whether an issue is about a bug.

Antoniol et al.~\cite{antoniol2008bug} investigated the automatic classification of issue reports
by utilizing conventional text mining techniques based on the description part of issue reports.
By extracting the textual part of issue reports (title, description, and discussions) from ITS of three case projects
and building classifiers using three supervised MLTs
(alternating decision trees, Naive Bayes classifiers, and logistic regression),
linguistic information in ITS is sufficient (82\% best precision for three case projects) to automatically
distinguish bugs from other activities.

Zhou et al.~\cite{zhou2014combining} proposed a hybrid approach that combines text and data mining techniques 
and considers the misclassification of issue reports in ITS. 
They took advantage of structural information with textual information that proposed a hybrid approach 
that combines text and data mining techniques, 
and achieved an excellent result (average 84.7 for 5 case projects).

A common approach adds information extracted from ITS~\cite{antoniol2008bug,zhou2014combining,merten2016software} 
to improve the performance of the model. 
In~\cite{antoniol2008bug}, discussions are involved; in~\cite{zhou2014combining}, structural information, such as severity, priority, and component, are utilized; 
and in~\cite{merten2016software}, metadata extracted from ITS are proved to outperform classifiers. 
However, these kinds of data are not produced when the issue report is submitted. 
Consequently, obtaining structured information in GitHub is difficult because of omissions.
Textual summary is a main type of information used to build the classification model.

In addition, some researchers work on cold-start problem of building classification model.
The main idea of solving the problem is using cross-project dataset.
And the method is well used in works like bug prediction~\cite{zimmermann2009cross,zhang2016cross}.
Merten et al.~\cite{merten2016software} first explore to use the cross-project model to classify issue reports. 
They use the model of one project to classify issue reports of other project, and find trained models can be re-used for other projects.
Zeng et al.~\cite{yu2018transferring} propose a novel cross-project approach which integrate multiple models learned from various source projects to classify target project. 


% Concerning these, we propose an approach to build classification model only extracting title and description of issue report. 

% Moreover, mining feature requests from textual summary of user feedbacks is also what researchers are concerned about.

% Walid et al.~\cite{maalej2015bug} utilize MLT on feedback from app store.
% They combine metadata (i.e., star rating, tense, etc.) with natural language processing, and classify app reviews into four types: bug reports, feature requests, user experiences, and ratings.
% Finally, the classification precision got between 70-95\% while the recall between 80-90\%, and they found that multiple binary classifiers outperformed single multiclass classifier.

% Thorsten et al.~\cite{merten2016software} investigates natural language processing and machine learning features to detect software feature requests in natural language data of issue tracking systems.
% They compare traditional linguistic machine learning features, such as “bag of words”, with more advanced features, such as subject-action-object.
% And they find that request can be detected best out of the researched SFR parts.


% Researches introduced before have utilized ML techniques
% to automatically classify user feedbacks in dataset like Bugzilla and user comments in app store.
% Thinking about the huge difference between ITS of GitHub and other dataset,
% there are no one has studied whether these ML techniques are worked on the ITS of GitHub.
% So we further propose the first question:

% \textbf{RQ1: \textit{Is traditional ML techniques also applicable in ITS of GitHub?
% Which ML technique performs best for classifying issue report for most projects in GitHub?}}

% %Different issues are not same for machine learning method to classify.
% The performance of ML techniques are inconsistent among
% the issue reports with different features.
% (e.g., the length of description).
% Some issues are easy to distinguish the category of them,
% but others are hard to tell.
% Instead of directly giving the answer about bug or not a bug,
% Zhou et al.~\cite{zhou2014combining} classify the issues into
% three levels of likelihood of being a corrective bug.
% Actually, for many ML techniques,
% they can output probability of classification results.
% The probability reflects
% how we can trust the prediction results generated by a classification model.
% Then we ask:

% \textbf{RQ2: \textit{What factors affect an issue report to be right classified?}}

% After manually analyzing the patterns why some issues are difficult to automatically classify,
% a nature idea come out that can we utilize above patterns
% to improving accuracy of our classification models.
% So we ask:

% \textbf{RQ3: \textit{If patterns for these issues are exist,
% can we take advantage of it to improve performance of classification model?}}



%\yy{to be determined}
% Through topic model, we can aggregate issues according to topic of issues or the component that issues are involved into, etc. From topics of issues, the distribution of different categories of issues may be imbalanced, which means that there are more bugs appear in a topic and more features appear in another topic. if so, we can use the imbalanced distribution of issues to correct the classification model and improving the performance of classification model. So, we have the last question:

% \textbf{RQ4: \textit{Is precision of issues associate with topics of them?}}