Vagner F De Santana

IBM · Thomas J. Watson Research Center

Invited talk to the panel "LLMs on the Web: Anticipating the Challenge." https://www.intgovforum.org/en/content/igf-2023-ws-217-large-language-models-on-the-web-anticipating-the-challenge

Evaluating human-AI decision-making systems is an emerging challenge as new ways of combining multiple AI models towards a specific goal are proposed every day. As humans interact with AI in decision-making systems, multiple factors may be present in a task including trust, interpretability, and explainability, amongst others. In this context, this...

Agriculture is impacted by multiple variables such as weather, soil, crop, stocks, socioeconomic context, cultural aspects, supply and demand, just to name a few. Hence, understanding this domain and identifying challenges faced by stakeholders is hard to scale due to its highly localized nature. This work builds upon six months of field research a...

Technology development practices in industry are often primarily focused on business results, which risks creating unbalanced power relations between corporate interests and the needs or concerns of people who are affected by technology implementation and use. These practices, and their associated cultural norms, may result in uses of technology th...

Computer Anxiety is a phenomenon studied in multiple contexts and, in the actual COVID-19 scenario, it is gaining more and more importance as it impacts technology adoption and autonomy. People with Computer Anxiety (PwCA) might feel intimidated, afraid of feeling embarrassed or scared of damaging computers, even before the actual interaction. Thus...

Recommendation systems are present in multiple contexts as e-commerce, websites, and media streaming services. As scenarios get more complex, techniques and tools have to consider a number of variables. When recommending services/products to mobile users while they are in indoor environments next to the object of the recommendation, variables as lo...

Computer anxiety (CA) can be defined as fear and worries that someone may feel when using computers. Thus, people with CA may face problems when using computers at home, at work or for study purposes, resulting in multiple forms of barriers even before the actual interaction with computers. In this context, the purpose of this research is to identi...

Recent advances in Artificial Intelligence (AI) and Machine Learning (ML) brought light on the need for explainability in multiple domains (e.g., health-care, finance, justice, and recruiting). Explainability or Explainable AI (XAI) can be defined as everything that makes AI more understandable to human beings. However, XAI levels may vary accordin...

An online learning system selects content for a learning session and opens a user interface to start the learning session on a plurality of devices. A mode of instruction is selected for the learning session. An activity to perform associated with the content is presented. Performance of the activity is monitored and a performance metric and/or a h...

A method for identifying seismic horizons in digitized seismic images includes deploying a swarm of agents at an initial position in a seismic image to be analyzed, where the swarm of agents includes picking agents that define a direction for the swarm and averaging agents that smooth the direction of the swarm, identifying a direction to follow in...

A technique relates to a manufacturing process. A computer system receives a material identification of a resource being manufactured, and receives images captured of units of the resource, the resource having been cut into the units. Features are determined for each of the units from the images. Feature vectors are created on a one-to-one basis fo...

A system, method and program product for applying interventions to allocated resources intervention impacts identified and minimized. After receiving a change request requesting changes involving shared resources, elements affected by implementing the requested change are identified. Individual impacts to elements in each layer are determined momen...

Attributes associated with a farming area and production data may be received. A user's expertise level may be identified. At least one crop model may be executed to generate a plurality of scenarios specifying specific actions to perform associated with a farming topic. A production goal associated with the farming area may be received. Based on t...

A method for processing an attempted payment made using a mobile device includes receiving information about the attempted payment, receiving data indicative of a behavior of a user of the mobile device at the time of the attempted payment, computing a likelihood that the attempted payment is fraudulent, based on a comparison of the behavior of the...

The coronavirus pandemic forced all the world to find new ways to 'get the work done. ' Social isolation in particular shifts the work to primarily remote, which compelled us to reassess a particular set of work practices that were before inherently personal and high-touch in nature. Business-to-Business (B2B) sales practices have been already unde...

A user interaction with a computer device is monitored, the user interaction comprising at least user consuming material via the computer device. Content of the material the user is consuming is identified. Physiological pattern generated by the user while consuming the content is captured via at a sensor device. Responsive to detecting that the ph...

A method for indexing a user interface test recording includes capturing, during a session, a recording of a plurality of interactions with the user interface, capturing, during the session, an event stream including a plurality of user interface events, synchronizing, in time, the plurality of interactions and the plurality of user interface event...

A customized video stream is provided. A second content is searched for that includes the plurality of entities, and a plurality of user data associated with the social media account. A hierarchy of additional content is generated based, at least in part, on the second content that is associated with the plurality of entities, and the user data ass...

Method and system for analyzing temporal data includes obtaining a first data set of temporal data of a first size and transforming the temporal data into a computer generated temporal graph containing a set of predefined data properties. The method includes generating a time decay function of a selected temporal attribute and fitting the time deca...

Providing knowledge representation of material content being consumed by a user combines the user's current behavioral data and data from external sources such as internet web sites and social media network. Visual representations of entities and their relationships in the content being consumed by the user are created while the user is consuming c...

A user guided data mining system includes: a database storing a network graph comprising a plurality nodes and edges, node attributes for the nodes, and edge attributes for the edges; and a data mining module configured to: request feedback from a user indicating at least one of the node attributes, visualize the network graph considering the indic...

An online learning system selects content for a learning session and opens a user interface to start the learning session on a plurality of devices. A mode of instruction is selected for the learning session. An activity to perform associated with the content is presented. Performance of the activity is monitored and a performance metric and/or a h...

Methods, systems, and computer program products for recommending activity sensor usage by image processing are provided herein. A computer-implemented method includes identifying, based on (i) sensor data from one or more sensors during a user activity and (ii) video data associated with the user performing the user activity, positioning of the one...

Predicting invoice payment is valuable in multiple industries and supports decision-making processes in most financial workflows. However, the challenge in this realm involves dealing with complex data and the lack of data related to decisions-making processes not registered in the accounts receivable system. This work presents a prototype develope...

Com o aumento do uso de diferentes tipos de tecnologias no cotidiano, seria esperado que usar computadores fosse confortável para todos. Porém, algumas pessoas ainda sentem-se apreensivas antes e durante o uso de computadores, enfrentando múltiplos níveis de ansiedade e, para níveis elevados, demonstram o que é chamado de Ansiedade Computacional (d...

A system, method and program product for implementing a sketch-based retrieval system. A system is disclosed that includes: an interface having a canvas for entering inputs including user generated strokes, and extracting features associated with the inputs; a hierarchy system for assigning an importance to the inputs; a matching system for retriev...

Tooling user studies tasks is fundamental to reduce the burden of evaluators, practitioners, facilitators, and observers during user interface (UI) evaluations. However, available tools for collecting data beyond clicks are usually paid, count on limited vocabulary of UI events, have complex setup, or do not provide easy download of captured data....

Due to the fast growth of the Web and to the interactivity brought by Web 2.0, government, entertainment, and education services are more and more available through the Internet. However, this highly interactive Web still excludes people by not considering capabilities of all users. Although there are techniques and approaches towards Web Accessibi...

Even nowadays, the use of computers can be difficult for some people. The reason for that can be complex and multifactorial, such as lack of experience in using computers, lack of knowledge, problems related to usability and accessibility, fear, intimidation, or anxiety. Some people can experience multiple levels of anxiety and, for high ones, they...

Tooling user studies tasks is fundamental to reduce the burden of evaluators, practitioners, facilitators, and observers during user interface (UI) evaluations. However, available tools for collecting data beyond clicks are paid, count on limited capabilities for capturing/downloading large datasets, or have complex setup. In this context, we prese...

A method for processing an attempted payment made using a mobile device includes receiving information about the attempted payment, receiving data indicative of a behavior of a user of the mobile device at the time of the attempted payment, computing a likelihood that the attempted payment is fraudulent, based on a comparison of the behavior of the...

Identifying how geoscientists interact with seismic images allows a deeper understanding of their rationale and the seismic interpretation process itself. Moreover, identifying nuances involving seismic interpreters performing slightly different roles opens new possibilities relative to how decision support systems could become part of the seismic...

An automated feedback may be provided while data is entered in an electronic form. Input entered into an input field of a form fillable window provided on a user interface display screen may be detected. A quality measure associated with the input as entered into the form fillable window may be determined. The quality measure is determined based on...

A system, method and program product for applying interventions to allocated resources intervention impacts identified and minimized. After receiving a change request requesting changes involving shared resources, elements affected by implementing the requested change are identified. Individual impacts to elements in each layer are determined momen...

A notification including one or more policies of use of one or more devices in an environment is received. The one or more policies are generated based on a pattern of use detected based on interaction data obtained by one or more sensors in the environment. Operation of one or more functionalities in the one or more devices is adjusted based on th...

An automated feedback may be provided while data is entered in an electronic form. Input entered into an input field of a form fillable window provided on a user interface display screen may be detected. A quality measure associated with the input as entered into the form fillable window may be determined. The quality measure is determined based on...

An aspect of the disclosure includes a method, a system and a computer program product for planning and presenting a topic to a plurality of students. The method includes defining a topic for a plurality of students and a time period for presenting the topic to the plurality of students. Each of the students has an associated learning style, the le...

Interaction logs (or usage data) are abundant in the era of Big Data, but making sense of these data having Human-Computer Interaction (HCI) in mind is becoming a bigger challenge. Interaction Log Analysis involves tackling problems as automatic task identification, modeling task deviation, and computing task learning curve. In this work, we propos...

A method is provided, the method including: displaying an image on a display; detect a user input corresponding to one or more portions of the image; analyzing the user input to determine at least one feature vector corresponding to the user input; and determining a classification for the one or more portions of the image based at least on the at l...

A method and system for analyzing temporal data includes selecting a data source in which data from the source contains temporal attributes, obtaining temporal data of a first size from the selected data source and mapping the temporal data from the selected source in a temporal graph. The method and system selects a temporal attribute, generates a...

As diretrizes de acessibilidade de conteúdo (WCAG 2.0) do World Wide Web Consortium (W3C) completam neste ano 10 anos com o status de recomendação. Entretanto, a Web ainda conta com muitas barreiras relacionadas a problemas básicos de codificação. Estudo aponta que 85% dos websites apresentam problemas como falta de rótulos em formulários e de desc...

Presentation used in the short course about web development aiming at accessibility and usability.

Aspects of the present invention include a method, system and computer program product. The method includes a method includes selecting a chart, and interpreting contents within the selected chart. The method also includes searching for additional sources of information, and extracting information from the additional sources of information. The met...

Designing systems to offer support to experts during cognitive intensive tasks at the right time is still a challenging endeavor, despite years of research progress in the area. This paper proposes a gaze model based on eye tracking empirical data to identify when a system should proactively interact with the expert during visual inspection tasks....

Taxonomies are valuable for providing a standardized way of cataloging elements into categories. In the context of website evaluation tools, providing a structured way for researchers and practitioners to compare and analyze existing solutions is valuable for identifying gaps/trends or to support well-informed decisions during development cycles (f...

With the increasing use of technology in everyday life, one would expect that the use of computers would be comfortable and straight­forward for everyone. However, some people still feel intimidated when using computers. Those people experience multiple levels of anxiety and, for high levels of anxiety, manifest what is known as Computer Anxiety. P...

Health insurance companies in Brazil have their data about claims organized having the view only for service providers. In this way, they lose the view of physicians' activity and how physicians share patients. Partnership between physicians can be seen as fruitful, when they team up to help a patient, but could represent an issue as well, when a r...

Health insurance companies in Brazil have their data about claims organized having the view only for providers. In this way, they loose the physician view and how they share patients. Partnership between physicians can view as a fruitful work in most of the cases but sometimes this could be a problem for health insurance companies and patients, for...

Health insurance companies in Brazil have their data about claims organized having the view only for providers. In this way, they loose the physician view and how they share patients. Partnership between physicians can view as a fruitful work in most of the cases but sometimes this could be a problem for health insurance companies and patients, for...

O projeto Websites Atendendo a Requisitos de Acessibilidade e Usabilidade, ou WARAU, nasceu como um website que tinha o objetivo de ser um espaço de discussão sobre normas, diretrizes, técnicas e boas práticas para a criação de código Web acessível e usável. O nascimento do projeto foi marcado pela publicação do relatório técnico intitulado Um proc...

Detailed usage data is becoming available through different devices (e.g., personal computer, cell phones, tablets, watches, glasses, wrist bands), in huge volumes, and in a speed that requires new models and visualizations to support the understanding of detailed user actions at scale. Without appropriate methods that summarize or provide means of...

Grand Challenges are ranked and defined to guide an area/discipline so that efforts and long term initiatives focus on providing disruptive solutions. These challenges represent a key component to highlight core issues that cannot be tackled in a short term or by a single person, requiring multiple efforts and sometimes whole communities. This posi...

Logs de interação estão presentes nas mais diversas plataformas, de mainframes a dispositivos móveis. A análise de logs ganhou espaço com o advento do comércio eletrônico, em que o objetivo era analisar produtos mais comprados, origem de acessos ou até quando usuários desistiam de uma compra. Esta visão promoveu o surgimento de diversas ferramentas...

BACKGROUND Social protocols are a set of rules or policies for human interactions which lead to a well-coordinated collaboration among the members of a group in a particular environment. Social protocols may guide a person to behave in a certain manner so as to not be disruptive in an environment. Social protocols may also guide a person's use of...

Knowing the main characteristics of a scientific community, how it reaches all stakeholders, and understanding how individuals engage around a subject is needed in order to support decision makers to plan strategies to maintain and nurture the community. This work presents a new way of interpreting the reach of a scientific community by incorporati...

GRC (Governance, Risk, and Compliance) data is voluminous and highly interrelated, yet sparsely populated. This fact represents one of the biggest challenges when creating visualizations for such datasets: the data does not align well in a tabular structure typically used to populate displays and reports. GRC Platforms provide reporting capabilitie...

Care teams are formed by physicians of different specialties who take care of the same patient. Hence, if we find physicians that share patients with each other probably they configure an informal care team. Thus, the objective of this work is to explore the possibility of finding care teams using Social Network Analysis techniques in physician-phy...

Healthcare insurance data represent a rich source of information and has the potential to contribute signficantly in guiding business decision making. In this work we present GraPhys, a Graph Analysis platform designed for exploration, visualization and analysis of healthcare insurance data and its corresponding metadata. By taking advantage of rel...

Vocational training of people with disabilities (PwD) can potentially improve social and economic prospects, but at the same time, it can be significantly challenging due to the need for specialized training and technology. Unfortunately, in developing countries this problem is magnified because, in general, low- income groups have limited access t...

Internet offers a huge amount of services related to government, entertainment, and communication. However, to ensure that any citizen can have proper access to all services and information, it is necessary to think about accessibility in a broader way than it is found in the Web. Universal Design points to a way of thinking about the heterogeneity...

Healthcare insurance data represent a rich source of information and has the potential to contribute significantly in guiding business decision making. In this work we present GraPhys, a Graph Analysis platform designed for exploration, visualization and analysis of healthcare insurance data and its corresponding metadata. By taking advantage of re...

Identifying maximum/minimum values in different charts is a common goal when users are interacting with visualizations. Moreover, this can be a challenging task when the user is interacting with space-filling visualizations. Space-filling methods consider a 2D area to represent values, however, low values that might be interesting for decision make...

Although websites evaluation tools use different data sources (e.g., Web pages, server logs, and mouse tracks), few of them support remote evaluation using detailed observational data. Without considering data that represent the user's real interaction with the interface, usability problems and/or accessibility barriers may remain unknown. This wor...

Usability test videos are one of the results of usability tests. They are used for future reference, documentation, and detailed analysis. However, analysis of videos demands concentration and is time consuming. This paper contributes with a method to annotate usability test videos based on logs of UI events, indicating usability problems and provi...

Solving today’s problems demands more than the effort of an individual, however, brilliant mind. Collaboration and team work are fundamental skills for tackling such problems. The ability of team members to work together and communicate with one another thus becomes an uppermost concern. In this context, to assemble an effective team requires an ap...

Solutions for personalizing websites by automatically changing user interfaces (UI) to fit users’ needs have been proposed by the industry and the academy in order to provide individualized user experience. However, the users’ perception of changes in the tailored UI is still a topic to be studied. This work presents a tool developed to capture log...

In the past years, social network analysis (SNA) techniques got disseminated to a large number of fields, such as, disease spread, animal group behaviour, organization dynamics, and the like. A particular use for SNA is to help in understanding how organizations work. It focuses not only on issues relative to how to manage organizations, but how pe...