How can deep learning models in medical research achieve both accuracy and transparency, ensuring trust in critical decisions?

First of all, I am not an expert. My intention is to start a discussion topic so experts can place their opinions and initiate an exchange of knowledge. However, I will try to unfold my thoughts even though I may be completely wrong, so please be lenient. Once upon a time, the first ever topic/question I read in RG was about transparency of NNs. I realized that there were two sides of the story. Some were critizing NNs calling them "black-boxes" that will remain as that, and others saying that there are transparency enhancement techniques like "symbolic rules" and "rule extraction" approaches (I deeply apologize if I dont use the correct names of the fields) that can practically change in some way the way of realizing the NNs. Anyway, today there are interpretability techniques for visualization of features like SHAP/LIME that improve interpretability.

After this long intro, I am now trying to unfold my thoughts regarding the transparency of RL as a concept. At first, I do not consider to distinguish between Model-Free and Model-Based cases. Lets take DDPG or TD3 or other extensions that may include more complex cases from the perspective of the neural networks applied for actor and critic networks (2 sets or 3 sets of neural network pairs respectively). My thoughts are that even if those RL approaches may include Deep neural networks for policy or value approximation, inheriting challenges of transparency from deep learning, they still have the reward function in the environment model. In practice, I can formulate the reward function incorporating different objectives that may be complex and at the same time contradictive. Also, each objective is formulated in a way together with others to synthesize a penalization on the agent's decision. So from each objective's perspective there is one formulation that may be analytical or not and may not always have a direct real meaning but in any way it prompts the agent to go towards specific regions of the desired space. In my mind even though RL can use big neural networks for the critic and actor, the formulation of reward and the way the networks are trained seems to be more transparent as in Deep Learning (not fully transparent though). I could say, if I may, transparent enough as I can plot the different components that formulate the total reward to see their behavior. Of course it is not that transparent how those individual penalty factors are synthesized together prompting the agent.

Therefore, if you agree, the reward function has a central role in RL by defining the agent's objectives. Even if it goes complex or even if it involves conflicting objectives, each component/penalty factor of reward is kind of interpretable. This way, I can understand why the agent is incentivized to make specific actions/decisions.

So now, even if you agree about the aforementioned central role of reward function in RL, a dilemma emerges in my mind. My question revolves around whether RL, as a concept, retains its level of transparency when Deep NNs are incorporated into its framework and whether the centrality of the reward function outweights the opacity introduced by these networks. Could we rank which is more important? The reward's central role (if such) or the inclusion of DNNs? From the perspective of DNNs, does the inclusion of DNNs make RL inherently less transparent? For the latter question my first respond would be "No", because i cannot understand how the DNN affects the reward. On the other hand, someone would say that the learned policies and value functions are represented in high-dimensional spaces and the relationship between state and action becomes less intuitive, so reward function does not "alleviate" transparency much.

Maybe no one is interested in those questions, or maybe the answers are very clear to everyone but me. If anyone does find these questions worth exploring, I’d be very thankful for your time and any input you’re willing to share. For sure there are people here who know much more about this and can provide clarity or help me see things from a "clear" perspective. If anyone is aware of studies or papers that address these questions, I would greatly appreciate it if you could mention them here. It would be helpful to learn from existing research and make such works more widely known. Still, I hope this sparks some discussion...

Increasing AI interpretability is crucial for understanding how decisions are made, especially in high-stakes domains like healthcare and finance.

ANN/Deep Learning/CNN etc., are they not overhyped? Today under the sun they promise to deliver solution to every problem. But in practise they have limitations. Besides this they are not transparent in decisions. Convergence is a major issue. Continuous learning requires retraining, which is cumbersome. To me it looks they are overhyped. Other algorithms also perform equally well in decision making.

In the first place, this question is not about encoding, or feature representation. We are aware of the important role given to object features in various visual scenarios. It covers a wide spectrum in vision that at least includes, salience maps, object relevance, eye movements, attention process, fixation, etc. It is really a hard problem!! of not encountering some form of feature-based visual model. Although, with exception of our recent research proposal, Entropy Driven Deep Learning, attempting to introduce an alternative view. Let us, get back to the challenging dilemma of characterizing object features, thus enable to evaluate its contribution to system objective(s). Hence, we may characterize features by the contribution. Then, other immediate problem arise that concern how to encode feature contributions and characterize features as separate entities! Well, it seems the feature evaluation is a problem especially in proactive vision. As well as higher cognitive visual processes such as perceptual organization.We are addressing the problem in our ongoing research proposal. Your response to the posed problem will help many research works including ours.

In the context of machine learning models for healthcare that predominantly handle discrete data and require high interpretability and simplicity, which approach offers more advantages:

Rough Set Theory or Neutrosophic Logic?

I invite experts to share their insights or experiences regarding the effectiveness, challenges, and suitability of these methodologies in managing uncertainties within health applications.

The quality of the formation and control of the implementation of the current and forward-looking socio-economic policy of the state and/or a local government unit is determined not only by strictly factual considerations concerning the specific conditions and determinants of the country's economic development, the phase of the business cycle in which the economy is, the level of economic growth and prosperity in individual accounting periods in relation to the planned economic policy and the draft state budget, the prosperity of the global economy and international economic relations, etc., but also by the level of reliability, implementation of the objectives of the state and/or a local government unit. but also the level of reliability, achievement of objectives, consideration of certain principles, including public service ethics, reduction of the scale of corruption and embezzlement of public funds, etc. Bearing in mind the aforementioned issues, which significantly affect the quality of implementation of a socio-economic policy programme planned for a specific period of time, it is possible to engage artificial intelligence to shape and control the implementation of current and prospective socio-economic policy of the state and/or local government unit. Of course, it would be up to humans to accept the final shape of the socio-economic policy programme proposal designed by artificial intelligence and to control its implementation. But in a situation of an appropriate increase in transparency and fuller implementation of public functions, the involvement in the role of a multi-criteria, intelligent system designing specific socio-economic policy programmes and their implementation and control of their implementation could significantly increase the effectiveness of the overall economic policy of the state, including issues of reliability, public transparency, improving the process of building a welfare economy. In the process of shaping specific socio-economic policy programmes, in addition to artificial intelligence, business analytics implemented on computerised analytical platforms Business Intelligence and Big Data Analytics may be involved, as well as, as required, other information technologies of the current fourth technological revolution, Industry 4.0 technology. The aforementioned new technologies can also be particularly helpful in developing forecasting models for estimating current and prospective levels of systemic financial risks, economic risks, the indebtedness of the state's public finance system, systemic credit risks of commercially operating financial institutions and economic entities and forecasting future financial and economic crises. Data on the aforementioned systemic risks and forecasting of trends in the economic situation can be helpful for planning the shape and implementation instruments of socio-economic policy for the following quarters and years. Besides, artificial intelligence technologies and other Industry 4.0 technologies may also help in the precise determination of the appropriate budgetary, fiscal and monetary policy instruments for designing interventionist, anti-crisis, pro-development, Keynesian socio-economic policy programmes, i.e. programmes precisely tailored to certain diagnosed determinants shaping the current and prospective economic situation.

In view of the above, I address the following question to the esteemed community of scientists and researchers:

What do you think about this topic?

What is your opinion on this subject?

Please respond,

I invite you all to discuss,

Thank you very much,

Best wishes,

Dariusz Prokopowicz

I was going through some surveys discussing explainability in deep learning models. I realized the authors use Explainable AI, Interpretable AI, and Transparent AI interchangeably.

So, I was wondering whether they are all the same or if there is any difference among the terms.

Hello,

you can find attached a photo where colonies of both, wt strain and ko mutant (mut), are shown. A gene encoding a key protein probably involved in CPS synthesis was deleted by double crossover.

They are at the same concentration, but the mutant did not present the typical heterogenous phenotype of the strain, since no transparent colonies were seen.

Any suggestion about the function of the lacking gene? I would appreciate any help because this is not my field.

Thanks!