Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach

Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing

Player FM - Internet Radio Done Right

otuz üç hafta önce eklendi

İçerik Risk Insights: Yusuf Moolla tarafından sağlanmıştır. Bölümler, grafikler ve podcast açıklamaları dahil tüm podcast içeriği doğrudan Risk Insights: Yusuf Moolla veya podcast platform ortağı tarafından yüklenir ve sağlanır. Birinin telif hakkıyla korunan çalışmanızı izniniz olmadan kullandığını düşünüyorsanız burada https://tr.player.fm/legal özetlenen süreci takip edebilirsiniz.

Threshold

1
Hark | 8 | The Queen's English 30:00

17 gün önce30:00

Daha Sonra Çal

Listeler

Beğen

Beğenildi

30:00

Living together in a group is a strategy many animals use to survive and thrive. And a big part of what makes that living situation successful is listening. In this episode, we explore the collaborative world of the naked mole-rat. Threshold is nonprofit, listener-supported, and independently produced. You can support Threshold by donating today . To stay connected, sign up for our newsletter . Operation frog sound! Send us your frog sounds for an upcoming episode. We want you to go out, listen for frogs and toads, and record them. Just find someone croaking, and hit record on your phone. It doesn’t matter if there’s background noise. It doesn’t even matter if you’re not sure whether or not you’re hearing an amphibian—if you think you are, we would love to get a recording from you. Please also say your name and where you are in the world, and then email the recording to us at outreach@thresholdpodcast.org…

1 yıl önce 8:07

MP3•Bölüm sayfası

Spoken (by a human) version of this article.

Outcome-focused accuracy reviews directly verify results, offering more robust assurance than process-focused methods.
This approach can catch translation errors, unintended consequences, and edge cases that process reviews might miss.
While more time-consuming and complex, outcome-focused reviews provide deeper insights into system reliability and accuracy.

This article explains why verifying outcomes is preferred over tracing through processes, and how it works.

About this podcast
A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI.
Hosted by Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

27 bölüm

#MBA #Business #Risk Insights: Yusuf Moolla #Audit #Algorithms #Bias #Accuracy #Fair #Algorithm Governance #Algorithm Integrity #Ai Governance #Ai Assurance #Ai Audit

Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach

Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing

published 1 yıl önce

Paylaş

MP3•Bölüm sayfası

Spoken (by a human) version of this article.

Outcome-focused accuracy reviews directly verify results, offering more robust assurance than process-focused methods.
This approach can catch translation errors, unintended consequences, and edge cases that process reviews might miss.
While more time-consuming and complex, outcome-focused reviews provide deeper insights into system reliability and accuracy.

This article explains why verifying outcomes is preferred over tracing through processes, and how it works.

27 bölüm

#MBA #Business #Risk Insights: Yusuf Moolla #Audit #Algorithms #Bias #Accuracy #Fair #Algorithm Governance #Algorithm Integrity #Ai Governance #Ai Assurance #Ai Audit

Tüm bölümler

1
Article 23. Algorithmic System Integrity: Testing 5:47

7 weeks önce5:47

5:47

Spoken by a human version of this article. TL;DR (TL;DL?) Testing is a core basic step for algorithmic integrity. Testing involves various stages, from developer self-checks to UAT. Where these happen will depend on whether the system is built in-house or bought. Testing needs to cover several integrity aspects, including accuracy, fairness, security, privacy, and performance. Continuous testing is needed for AI systems, differing from traditional testing due to the way these newer systems change (without code changes). About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 22. Algorithm Integrity: Third party assurance 7:26

8 weeks önce7:26

7:26

Spoken by a human version of this article. One question that comes up often is “How do we obtain assurance about third party products or services?” Depending on the nature of the relationship, and what you need assurance for, this can vary widely. This article attempts to lay out the options, considerations, and key steps to take. TL;DR (TL;DL?) Third-party assurance for algorithm integrity varies based on the nature of the relationship and specific needs, with several options. Key factors to consider include the importance and risk level of the service/product, regulatory expectations, complexity, transparency, and frequency of updates. Standardised assurance frameworks for algorithm integrity are still emerging; adopt a risk-based approach, and consider sector-specific standards like CPS230(Australia). About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Guest 3. Shea Brown, Founder and CEO of BABL AI 41:23

10 weeks önce41:23

41:23

Navigating AI Audits with Dr. Shea Brown Dr. Shea Brown is Founder and CEO of BABL AI BABL specializes in auditing and certifying AI systems, consulting on responsible AI practices, and offering online education. Shea shares his journey from astrophysics to AI auditing, the core services provided by BABL AI including compliance audits, technical testing, and risk assessments, and the importance of governance in AI. He also addresses the challenges posed by generative AI, the need for continuous upskilling in AI literacy, and the role of organizations like the IAAA and For Humanity in building consensus and standards in AI auditing. Finally, Shea provides insights on third-party risks, in-house AI developments, and key skills needed for effective AI governance. Chapter Markers 00:00 Introduction to Dr. Shea Brown and BABL AI 00:36 The Journey from Astrophysics to AI Auditing 02:22 Core Services and Compliance Audits at BABL 03:57 Educational Initiatives and AI Literacy 05:48 Collaborations and Professional Organizations 08:57 Approach to AI Audits and Readiness 17:29 Challenges with Generative AI in Audits 29:21 Trends in AI Deployment and Risk Assessment 34:53 Skills and Training for AI Governance 40:15 Conclusion and Contact Information About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 21. AI Risk Training: Role-based tailoring 6:26

10 weeks önce6:26

6:26

Spoken by a human version of this article. AI literacy is growing in importance (e.g., EU AI Act, IAIS). AI literacy needs vary across roles. Even "AI professionals" need AI Risk training. Links EU AI Act : The European Union Artificial Intelligence Act - specific expectation about “AI literacy”. IAIS: The International Association of Insurance Supervisors is developing a guidance paper on the supervision of AI . About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Guest 2. Patrick Sullivan: VP of Strategy and Innovation at A-LIGN 32:03

11 weeks önce32:03

32:03

Navigating AI Governance and Compliance Patrick Sullivan is Vice President of Strategy and Innovation at A-LIGN and an expert in cybersecurity and AI compliance with over 25 years of experience. Patrick shares his career journey, discusses his passion for educating executives and directors on effective governance, and explains the critical role of management systems like ISO 42001 in AI compliance. We discuss the complexities of AI governance, risk assessment, and the importance of clear organizational context. Patrick also highlights the challenges and benefits of AI assurance and offers insights into the changing landscape of AI standards and regulations. Chapter Markers 00:00 Introduction 00:23 Patrick's Career Journey 02:31 Focus on AI Governance 04:19 Importance of Education and Internal Training 08:08 Involvement in Industry Associations 14:13 AI Standards and Governance 20:06 Challenges with preparing for AI Certification 28:04 Future of AI Assurance About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Guest 1. Ryan Carrier: Executive Director of ForHumanity 44:49

12 weeks önce44:49

44:49

Mitigating AI Risks Ryan Carrier is founder and executive director of ForHumanity , a non-profit focused on mitigating the risks associated with AI, autonomous, and algorithmic systems. With 25 years of experience in financial services, Ryan discusses ForHumanity's mission to analyze and mitigate the downside risks of AI to benefit society. The conversation includes insights on the foundation of ForHumanity, the role of independent AI audits, educational programs offered by the ForHumanity AI Education and Training Center, AI governance, and the development of audit certification schemes. Ryan also highlights the importance of AI literacy, stakeholder management, and the future of AI governance and compliance. Chapter Markers 00:00 Introduction to Ryan Carrier and ForHumanity 00:57 Ryan's Background and Journey to AI 02:10 Founding ForHumanity: Mission and Early Challenges 05:15 Developing Independent Audits for AI 08:02 ForHumanity's Role and Activities 17:26 Education Programs and Certifications 29:21 AI Literacy and Future of Independent Audits 42:06 Getting Involved with ForHumanity About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 20. Algorithm Reviews: Public vs Private Reports 8:26

12 weeks önce8:26

8:26

Spoken (by a human) version of this article. Public AI audit reports aren't universally required; they mainly apply to high-risk applications and/or specific jurisdictions. The push for transparency primarily concerns independent audits, not internal reviews. Prepare by implementing ethical AI practices and conducting regular reviews. Note: High-risk AI systems in banking and insurance are subject to specific requirements Links AI and algorithm audit guidelines vary widely and are not universally applicable. We discussed this in a previous article , outlining how the appropriateness of audit guidance depends on your circumstances. Audit vs Review: we explored this topic in depth in a previous article . About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 19. Algorithmic System Reviews: Substantive vs. Controls Testing 6:26

13 weeks önce6:26

6:26

Spoken by a human version of this article. Knowing the basics of substantive testing vs. controls testing can help you determine if the review will meet your needs. Substantive testing directly identifies errors or unfairness, while controls testing evaluates governance effectiveness. The results/conclusions are different. Understanding these differences can also help you anticipate the extent of your team's involvement during the review process. Links This article details a (largely) substantive testing method for accuracy reviews. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 18. Algorithm Integrity: Training and Awareness 4:07

17 weeks önce4:07

4:07

Spoken by a human version of this article. Ongoing education helps everyone understand their role in responsibly developing and using algorithmic systems. Regulators and standard-setting bodies emphasise the need for AI literacy across all organisational levels. Links ForHumanity - join the growing community here . ForHumanity - free courses here . IAIS: The International Association of Insurance Supervisors is developing a guidance paper on the supervision of AI . DNB: De Nederlandsche Bank - 6 general principles for the use of AI in the financial sector . ASIC: The Australian Securities & Investments Commission - report . NIST: The National Institute of Standards and Technology - AI Risk Management Framework . EU AI Act : The European Union Artificial Intelligence Act - specific expectation about “AI literacy”. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 17. Algorithm Integrity: Audit vs Review 9:10

18 weeks önce9:10

9:10

Spoken by a human version of this article. The terminology – “audit” vs “review” - is important, but clarity about deliverables is more important when commissioning algorithm integrity assessments. Audits are formal, with an opinion or conclusion that can often be shared externally. Reviews come in various forms and typically produce recommendations, for internal use. Regardless of the terminology you use, when commissioning an assessment, clearly define and document the expected deliverable, including the report content and intended distribution, to ensure expectations are met. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach 8:07

19 weeks önce8:07

8:07

Spoken (by a human) version of this article. Outcome-focused accuracy reviews directly verify results, offering more robust assurance than process-focused methods. This approach can catch translation errors, unintended consequences, and edge cases that process reviews might miss. While more time-consuming and complex, outcome-focused reviews provide deeper insights into system reliability and accuracy. This article explains why verifying outcomes is preferred over tracing through processes, and how it works. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 15. Algorithm Integrity Documentation - Getting Started 5:19

20 weeks önce5:19

5:19

Spoken (by a human) version of this article. Documentation makes it easier to consistently maintain algorithm integrity. This is well known. But there are lots of types of documents to prepare, and often the first hurdle is just thinking about where to start. So this simple guide is meant to help do exactly that – get going. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 14. External data - use with care 6:59

21 weeks önce6:59

6:59

Spoken (by a human) version of this article. Banks and insurers are increasingly using external data; using them beyond their intended purpose can be risky (e.g. discriminatory). Emerging regulations and regulatory guidance emphasise the need for active oversight by boards, senior management to ensure responsible use of external data. Keeping the customer top of mind, asking the right questions, and focusing on the intended purpose of the data, can help reduce the risk. Law and guideline mentioned in the article: Colorado's External Consumer Data and Information Sources (ECDIS) law New York's proposed circular letter . About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 13. Bridging the purpose-risk gap: Customer-first algorithmic risk assessments 7:18

22 weeks önce7:18

7:18

Spoken (by a human) version of this article . Banks and insurers sometimes lose sight of their customer-centric purpose when assessing AI/algorithm risks, focusing instead on regular business risks and regulatory concerns. Regulators are noticing this disconnect. This article aims to outline why the disconnect happens and how we can fix it. Report mentioned in the article: ASIC, REP 798 Beware the gap: Governance arrangements in the face of AI innovation. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

1
Article 12. Risk-Focused Principles for Change Control in Algorithmic Systems 12:00

23 weeks önce12:00

12:00

Spoken (by a human) version of this article . With algorithmic systems, an change can trigger a cascade of unintended consequences, potentially compromising fairness, accountability, and public trust. So, managing changes is important. But if you use the wrong framework, your change control process may tick the boxes, but be both ineffective and inefficient. This article outlines a potential solution: a risk focused, principles-based approach to change control for algorithmic systems. Resource mentioned in the article: ISA 315 guideline for general IT controls. About this podcast A podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla . Produced by Risk Insights (riskinsights.com.au).…

Player FM'e Hoş Geldiniz!

Player FM şu anda sizin için internetteki yüksek kalitedeki podcast'leri arıyor. En iyi podcast uygulaması ve Android, iPhone ve internet üzerinde çalışıyor. Aboneliklerinizi cihazlar arasında eş zamanlamak için üye olun.

500+ konuyu dinleyin

Bounty Paper Towels Quick Size, White, 16 Family Rolls = 40 Regular Rolls (Packaging May Vary)

Nespresso Capsules Vertuo, Variety Pack, Medium and Dark Roast Coffee, 30 Count Coffee Pods, Brews 7.8 oz.

TOLOWOBK Easter Basket Stuffers for Teens Kids Girls Toddler Easter Basket Essentials Religious Christian Gifts for Teen Girls Cross Bracelet Trendy Stuff Egg Fillers Jesus WWJD Bracelets Cathlioc

3mm Tiny CZ Screw on Flat Back Stud Earrings,Gold Plated Flat Back Cubic Zirconia Earrings for Helix Cartilage Tragus Earlobe Piercing Jewelry Gift for Women Men(3mm CZ, Gold)

Mars M&M'S, SKITTLES, SNICKERS & STARBURST Easter Basket Stuffers Variety Pack, 65 Ct Bulk Bag

Dixie Medium Paper Plates, 8.5 Inch, 90 Count, 2X Stronger*, Microwave-Safe, Soak-Proof, Cut Resistant, Disposable Plates For Everyday Breakfast, Lunch, & Dinner Meals

LUMIFY Redness Reliever Eye Drops, For Whiter & Brighter Looking Eyes, 0.08 Fl Oz (2.5 mL)

Birthstone Name Necklace Personalized Jewelry for Women Mother's Day Gifts for Mom Gold Plated Pendant Charm, Custom Jewelry Grandma Gift Birthday Personalized Gifts for New Moms -CN-BS-SH

Dinlemeye Değer Podcast'ler

1
Hark | 8 | The Queen's English 30:00

Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing « »
Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach

Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach

Dinlemeye Değer Podcast'ler

Tüm bölümler

1
Article 23. Algorithmic System Integrity: Testing 5:47

1
Article 22. Algorithm Integrity: Third party assurance 7:26

1
Guest 3. Shea Brown, Founder and CEO of BABL AI 41:23

1
Article 21. AI Risk Training: Role-based tailoring 6:26

1
Guest 2. Patrick Sullivan: VP of Strategy and Innovation at A-LIGN 32:03

1
Guest 1. Ryan Carrier: Executive Director of ForHumanity 44:49

1
Article 20. Algorithm Reviews: Public vs Private Reports 8:26

1
Article 19. Algorithmic System Reviews: Substantive vs. Controls Testing 6:26

1
Article 18. Algorithm Integrity: Training and Awareness 4:07

1
Article 17. Algorithm Integrity: Audit vs Review 9:10

1
Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach 8:07

1
Article 15. Algorithm Integrity Documentation - Getting Started 5:19

1
Article 14. External data - use with care 6:59

1
Article 13. Bridging the purpose-risk gap: Customer-first algorithmic risk assessments 7:18

1
Article 12. Risk-Focused Principles for Change Control in Algorithmic Systems 12:00

Player FM'e Hoş Geldiniz!

Bounty Paper Towels Quick Size, White, 16 Family Rolls = 40 Regular Rolls (Packaging May Vary)

Nespresso Capsules Vertuo, Variety Pack, Medium and Dark Roast Coffee, 30 Count Coffee Pods, Brews 7.8 oz.

TOLOWOBK Easter Basket Stuffers for Teens Kids Girls Toddler Easter Basket Essentials Religious Christian Gifts for Teen Girls Cross Bracelet Trendy Stuff Egg Fillers Jesus WWJD Bracelets Cathlioc

Dinlemeye Değer Podcast'ler

Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing « » Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach

Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach

Dinlemeye Değer Podcast'ler

Player FM'e Hoş Geldiniz!

Hızlı referans rehberi

Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing « »
Article 16. Algorithmic System Accuracy Reviews – Choosing the Right Approach