Loading...
HomeMy WebLinkAbout13. Receive update to the 2023 Asset Management Plan Page 1 of 195 Item 13.a. CENTRALSAN jdf A- hom CENTRAL CONTRA COSTA SANITARY DISTRICT December 21, 2023 TO: HONORABLE BOARD OF DIRECTORS FROM: KHAE BOHAN, ASSET MANAGEMENT PROGRAM ADMINISTRATOR REVIEWED BY: NITIN GOEL, OPERATIONS OPTIMIZATION DIVISION MANAGER GREG NORBY, DEPUTY GENERAL MANAGER - OPERATIONS ROGER S. BAILEY, GENERAL MANAGER SUBJECT: RECEIVE UPDATE TO THE 2023 ASSET MANAGEMENT PLAN One of Central San's goals for the Fiscal Year 2014-16 Strategic Plan is to maintain a reliable wastewater infrastructure by establishing an asset management program. To support that effort, Central San adopted a policy and committed to the development of its asset management program. To move toward a comprehensive asset management program, staff hired a consultant to develop an Asset Management Implementation Plan (AMI P)to set a roadmap with these elements: • Develop Asset Management Organization, Policies, and Key Strategies to support asset management; • Develop Asset Register with asset hierarchy and data standards and to complete asset register for Pumping Stations and Treatment Plant assets; • Develop a Plant Maintenance Strategy with Reliability Centered Maintenance Program; • Develop Business Process Mapping to provide step by step workflows and formalize the relationships between tasks and responsibilities at which decisions are made and levels of actions to take place; • Develop Knowledge Management Strategy to capture asset related history, performance, and future requirements; • Develop Condition Assessments as a framework for identifying imminent failure mode, protocol for condition assessment, and optimize the process; • Develop a Comprehensive Maintenance Program based on asset criticality and consequence of failure; • Identification of the most effective Preventative Maintenance (PM)tasks for Treatment Plant equipment and systems; • Integration with a Computerized Maintenance Management System (CMMS)to establish a repeatable program with documented processes and procedures; and • Implementation of Cityworks Asset Management System (Cityworks replaced two existing platforms, Accela and Mainsaver). December 21, 2023 Regular Board Meeting Agenda Packet- Page 235 of 461 Page 2 of 195 With these significant goals and the AMI P completed, the next step in the evolution of the Asset Management Program is to document the systems and processes, the current state of Central San's assets, and the strategic and dynamic implementation throughout the life cycle of assets. The Asset Management Plan is a primary example of Central San's commitment to the environment, innovation, optimization, and continuous improvement. The purpose of the Asset Management Plan is to document the following: • An overview of Central San's assets, summarizing the current condition of assets and their overall performance; • System operation and maintenance strategies and documented procedures; • Level of service standards, including performance indicators and adopted targets; • Proposed actions to implement the above operation, maintenance, and strategies; and • Stakeholder engagement and communication plan. An Asset Management Policy, originally adopted in 2014, is under its biennial review as an item on the Administration Committee agenda. It will be updated to provide reference to this proposed Asset Management Plan and commits to being updated at least every five years. The Engineering and Operations Committee reviewed this matter at its April 3, 2023 meeting and recommended to bring this item to the Board with the Information/Operational Technology Strategic Plan which was presented recently. The Asset Management Program requires annual funding from the Operations and Maintenance Budget and the Capital Improvement Project Budget; both of these documents are presented separately to the Board for approval. Strategic Plan Tie-In GOAL SIX.Infrastructure Reliability Strategy 1 - Manage assets optimally, Strategy 2- Execute long-term capital renewal and replacement program ATTACHMENTS: 1.Asset Management Plan 2. Presentation December 21, 2023 Regular Board Meeting Agenda Packet- Page 236 of 461 E- v . : v VOW AD JT ASSET MANAGEMENT PLAN �CSDECEMBER 2023 CENTRAL CONTRA COSTA CENTRAL SAN SANITARY DISTRICT December 21, 2023 --. 461 Page 4 of 195 Disclaimer. While the Asset Management Plan document is for guidance and information, at times there may be revisions to the procedures, processes, and systems described in this living document. 2 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 238 of 461 Page 5 of 195 TABLE OF CONTENTS DOCUMENT VERSION HISTORY........................................................................................ 14 LIST OF SOFTWARE ........................................................................................................... 15 LISTOF ACRONYMS........................................................................................................... 15 EXECUTIVE SUMMARY....................................................................................................... 18 QUICK GLANCE AT THE ASSET MANAGEMENT PLAN .................................................. 19 BACKGROUND .................................................................................................................... 21 1. Introduction................................................................................................................. 21 2. Vision and Mission...................................................................................................... 22 A. Organizational Structure.......................................................................................22 B. History ..................................................................................................................24 C. Community Profile ................................................................................................24 3. Services Overview...................................................................................................... 25 A. Collection System.................................................................................................25 B. Wastewater Treatment .........................................................................................25 C. Recycled Water....................................................................................................26 D. Household Hazardous Waste Collection Facility ..................................................26 E. Pharmaceutical Collection ....................................................................................26 ASSET MANAGEMENT ORGANIZATIONAL CONTEXT.................................................... 28 1. Purpose ...................................................................................................................... 28 A. What is Asset Management?................................................................................28 B. Board Policy .........................................................................................................29 C. Asset Management Implementation Plan (2015)..................................................30 2. Governance ................................................................................................................ 30 A. Executive Leadership Support..............................................................................31 3. Compliance and Regulatory Requirements ................................................................ 32 A. Collection System.................................................................................................32 CONTEXT OF THE ORGANIZATION................................................................................... 33 1. Levels of Service ........................................................................................................ 33 A. Strategic Plan Summary.......................................................................................33 2. Strategic Alignment..................................................................................................... 36 A. Alignment with Mission, Vision, and Values..........................................................36 B. Alignment with Strategic Plan ...............................................................................37 3 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 239 of 461 Page 6 of 195 C. Alignment with Financial Plan (Budget), Capital Improvement Plan, and Technology Master Plan ..............................................................................................38 3. Stakeholder Engagement and Communication Plan .................................................. 39 A. Stakeholder Engagement .....................................................................................39 B. Internal Stakeholders............................................................................................39 C. External Stakeholders...........................................................................................41 D. Stakeholder Communication Plan.........................................................................41 ASSET DESCRIPTIONS....................................................................................................... 43 1. Existing Facilities ........................................................................................................ 43 A. Gravity System .....................................................................................................43 B. Force Main System...............................................................................................43 C. Pumping Stations..................................................................................................44 D. Treatment Plant....................................................................................................44 E. Recycled Water ....................................................................................................45 F. General Improvements .........................................................................................45 2. State of the Assets...................................................................................................... 45 A. Collection System.................................................................................................45 B. Treatment Plant....................................................................................................48 C. Recycled Water ....................................................................................................49 D. General Improvements .........................................................................................50 E. Capital Improvement Plan Prioritization — Linear Assets ......................................50 F. Capital Improvement Plan Prioritization — Vertical Assets ....................................51 ASSET MANAGEMENT INFORMATION SYSTEMS AND TOOLS ..................................... 53 1. Data and Knowledge Management............................................................................. 53 A. GeoPortal .............................................................................................................53 B. Mapping Customization for Field Staff..................................................................56 C. Map Updates ........................................................................................................56 D. Asset Inventory: Single Asset Register.................................................................56 E. Asset Hierarchies..................................................................................................57 F. Process Hierarchy ................................................................................................58 G. Data Management Strategies...............................................................................60 H. Computerized Maintenance Management System ...............................................62 I. Closed-Circuit Television ................................... ................65 ................................... J. Automatic Vehicle Locating ..................................................................................65 K. Underground Service Alert Locate Tickets ...........................................................65 L. Spare Parts Management Application ..................................................................65 4 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 240 of 461 Page 7 of 195 M. Reliability Engineering Tool ..................................................................................67 N. Document Management System ..........................................................................67 O. Modeling Software................................................................................................68 RISKS ................................................................................................................................... 72 1. Risk Definition............................................................................................................. 72 2. Identifying Risks — Condition Assessment.................................................................. 72 3. Quantifying the Risk.................................................................................................... 74 A. Probability of Failure, Consequence of Failure, and Business Risk Exposure......74 B. Consequence of Failure Scoring...........................................................................75 C. Business Risk Exposure Heat Map.......................................................................76 D. Aggregate Condition Assessment Scoring ...........................................................76 4. Condition Assessment Program ................................................................................. 79 A. Collection System.................................................................................................79 B. Treatment Plant, Pumping Stations, and General Improvements.........................85 C. Recycled Water....................................................................................................89 D. Asset Cost ............................................................................................................89 E. Replacement Cost................................................................................................89 5. Executive Level Risk Indices ...................................................................................... 90 DECISION-MAKING CRITERIA: DRIVERS AND LIFE CYCLE MANAGEMENT................ 92 1. Drivers: Aging Infrastructure, Regulatory, Sustainability and Optimization ................. 92 2. Capital Improvement Program .................................................................................... 92 A. Project Prioritization..............................................................................................93 B. Treatment Plant....................................................................................................93 C. Project Classification ............................................................................................94 3. Asset Lifecycle............................................................................................................ 94 A. Business Case Evaluation ....................................................................................95 B. Planning/Design....................................................................................................95 C. Construction: Create/Acquire................................................................................96 D. Asset Handover....................................................................................................96 E. Operate, Maintain, Modify, and Upgrade ..............................................................96 F. Remove and/or Replace.......................................................................................96 4. Remaining Useful Life................................................................................................. 97 5. Maintenance History................................................................................................... 98 6. Data Management and Utilization for Decision Making............................................... 99 A. Dashboards and Analytics....................................................................................99 5 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 241 of 461 Page 8 of 195 B. Machine Learning ...............................................................................................102 C. Smart Motor Control Centers..............................................................................103 ASSET MANAGEMENT APPROACHES ........................................................................... 104 1. Operations Strategies ............................................................................................... 104 A. Collection System — Pumping Stations...............................................................104 B. Treatment Plant..................................................................................................104 C. Recycled Water Distribution ...............................................................................104 2. Maintenance Strategies ............................................................................................ 104 3. Maintenance Procedures.......................................................................................... 106 A. Treatment Plant, Pumping Stations, and General Facilities................................106 B. Recycled Water ..................................................................................................106 4. Collection System ..................................................................................................... 109 A. Sewer System Management Plan.......................................................................109 B. Collection System...............................................................................................109 C. Preventative Maintenance ..................................................................................109 D. Corrective Maintenance......................................................................................110 E. Scheduling — Cleaning Optimization Application ................................................110 5. Treatment Plant, Pumping Stations, and General Improvements............................. 110 A. Reliability Centered Maintenance .......................................................................113 B. Preventive/Planned Maintenance Optimization ..................................................113 C. Defect Elimination - Also Known as Don't Just Fix It, Improve It ........................113 D. Example — Grease Gun Color Code Markings....................................................116 E. Example — Return Activated Sludge Pump Seals...............................................116 F. Root Cause Analysis ..........................................................................................116 G. Safety, Quality, and Effectiveness Observation..................................................117 H. Risk Threshold Investigation...............................................................................117 I. Failure Reporting Analysis and Corrective Action System..................................118 J. Reliability Engineering ........................................................................................121 K. Predictive and Asset Condition Monitoring Program...........................................124 L. Planning and Scheduling ....................................................................................127 M. Maintenance Involvement in Capital Projects.....................................................128 N. Building Information Modeling ............................................................................129 6. Recycled Water Distribution...................................................................................... 129 7. Standard Operating Procedures ............................................................................... 130 A. Plant Operations Division ...................................................................................130 B. Plant Maintenance Division ................................................................................130 6 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 242 of 461 Page 9 of 195 C. Pumping Stations Section...................................................................................130 8. Monitoring and Managing System Breakdowns........................................................ 130 A. Collection System...............................................................................................130 B. Treatment Plant and Pumping Stations ..............................................................130 C. Recycled Water Distribution ...............................................................................131 D. General Improvements .......................................................................................131 9. Updating and Maintaining Procedures...................................................................... 131 A. Collection System and Recycled Water Distribution ...........................................131 B. Treatment Plant, Pumping Stations, and General Facilities................................132 10. Performance Monitoring and Reporting .................................................................... 132 11. Operational Resilience and Disaster Preparedness ................................................. 136 A. Pumping Stations................................................................................................136 B. Public Safety Power Shutoffs .............................................................................137 C. Pandemic Response...........................................................................................137 STAFF DEVELOPMENT..................................................................................................... 139 1. Personnel Advancement........................................................................................... 139 2. Educational Resources............................................................................................. 141 3. Expanding the Internship Program ........................................................................... 142 4. Mentorship Program ................................................................................................. 142 5. Externship Program .................................................................................................. 143 6. Management and Supervisory Academy.................................................................. 143 7. Career Development Program .................................................................................. 144 8. Performance Recognition, Appreciation, and Beyond .............................................. 144 9. Team Accomplishments............................................................................................ 146 2023-2028 ASSET MANAGEMENT PLAN ELEMENTS .................................................... 147 1. Learning Takeaways................................................................................................. 147 2. Asset Management Plan — Goals for the Next 5 Years............................................. 149 3. Element Descriptions................................................................................................ 150 A. Element 1 — Complete the Asset Registry for Treatment Plant Piping Underground Utility..........................................................................................................................150 B. Element 2 — Continue the Development of Data Analytics .................................150 C. Element 3 — Continue Comprehensive Asset Evaluation for the Prioritizations of VerticalAssets...........................................................................................................151 D. Element 4 — Digitize Operator Round Sheets.....................................................151 E. Element 5 — Smart Utility/Digital Twin Initiatives.................................................151 7 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 243 of 461 Page 10 of 195 F. Element 6 — Dynamic Asset Management Plan..................................................152 GLOSSARY ........................................................................................................................ 153 REFERENCES.................................................................................................................... 159 APPENDICES ..................................................................................................................... 161 1. Board Policy 015, Asset Management...................................................................... 161 2. What is Asset Management?.................................................................................... 162 3. Service Area and Assets........................................................................................... 163 4. Asset Management Organization Chart.................................................................... 164 5. Recycled Water Distribution...................................................................................... 165 6. Cityworks Custom Business Process Manual........................................................... 166 7. Engineering Department Projects Procedures Manual ............................................. 169 8. Collection System Asset Handover Workflows ......................................................... 170 9. Treatment Plant Asset Handover Workflow .............................................................. 174 10. Asset Management Procedures Manual................................................................... 175 11. Business Case Evaluation ........................................................................................ 177 12. Pumping Stations Consequence of Failure Matrix.................................................... 182 13. Condition Assessment Results and Service Life....................................................... 183 14. Enterprise Risk Index................................................................................................ 184 15. Development of the Enterprise Risk Inventory.......................................................... 184 8 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 244 of 461 Page 11 of 195 TABLE OF FIGURES Figure 1: Asset Management and Reliability Engineering Timeline....................................... 18 Figure 2: Central San Service Area by the Numbers............................................................. 21 Figure 3: Central San in Relation to California and Nevada .................................................. 21 Figure 4: Headquarters, Treatment Plant, and HHW Collection Facility................................ 21 Figure 5: Collection System Operations Division Buildings ................................................... 21 Figure6: Cleaning ................................................................................................................. 22 Figure 7: Central San Organization Structure........................................................................ 23 Figure 8: Effective Utility Management Framework ............................................................... 34 Figure 9: Effective Utility Management Attributes.................................................................. 35 Figure 10: Central San Mission, Vision, and Values.............................................................. 36 Figure 11 : CSO Crew Cleaning a Sewer Main with a Vactor Truck....................................... 43 Figure 12: A Central San Pumping Station............................................................................ 44 Figure 13: Condition of the Gravity Portion of the Collection System .................................... 45 Figure 14: Capital Improvement Plan Prioritization Screenshot ............................................ 51 Figure 15: Conceptual Process Map for CIP Prioritization for Vertical Assets....................... 52 Figure 16: GIS Platforms, Infrastructure, and Data................................................................ 53 Figure 17: GeoPortal Web Application .................................................................................. 54 Figure 18: GIS Additional Details for Asset ........................................................................... 55 Figure 19: Schematic of Process Hierarchy and Geographic Relationship ........................... 57 Figure 20: Sample of an Asset Report for a Pumping Station ............................................... 58 Figure 21 : Sample Report for Assets in the Lab.......................................... .......... 58 ................ Figure 22: Process Hierarchy................................................................................................ 59 Figure 23: Geographical Relationship for the Ash Vac Blower 1 ........................................... 60 Figure 24: CMMS Screenshot of a Work Order and Associated Map.................................... 62 Figure 25: CMMS Screenshot of Workgroup Inbox ............................................................... 63 Figure 26: CMMS Screenshot of Mapped Work Orders ........................................................ 64 Figure 27: Spare Parts Search .............................................................................................. 66 Figure 28: Spare Parts Storage Locator................................................................................ 66 9 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 245 of 461 Page 12 of 195 Figure 29: Reliability Engineering Report Tool ...................................................................... 67 Figure 30: Visual Hydraulics Software of Central San Hydraulic Capacity Simulation ........... 69 Figure 31 : Sampling Locations on Process Flow Diagram .................................................... 71 Figure 32: Condition Assessment Levels .............................................................................. 73 Figure 33: Condition Assessment Phases............................................................................. 74 Figure 34: Heat Map of Risk as a Result of COF and POF ................................................... 76 Figure 35: CCTV Release Process........................................................................................ 81 Figure 36: Collection System Condition Assessment Program ............................................. 82 Figure 37: Risk Pyramid for Condition Assessment of Large Diameter Collection System Pipes ..................................................................................................................................... 83 Figure 38: Risk Pyramid for Condition Assessment of Force Mains...................................... 84 Figure 39: Inspection of Secondary Process High Value Assets and Support Equipment Informed Capital Projects Phasing ........................................................................................ 85 Figure 40: Inspection of Steam Process High Value Assets and Support Equipment Informed CapitalProjects Phasing ....................................................................................................... 86 Figure 41 : Central San Uses Advanced, Non-Invasive Technologies to Perform Condition Assessments and Minimize Operational Disruption............................................................... 87 Figure 42: Sample Asset Condition Report ........................................................................... 88 Figure 43: Key Drivers for Long-Term Planning .................................................................... 92 Figure44: Asset Lifecycle ..................................................................................................... 94 Figure 45: Anticipated Remaining Years of Useful Life ......................................................... 98 Figure 46: Equipment Failure Rate "Bathtub" Curve.............................................................. 99 Figure 47: CSO Cleaning Dashboard Showing Scheduled and Unscheduled Activities...... 100 Figure 48: CSO Preventative Maintenance Scheduled Weekly Dashboard with Indicators and Backlog Linked to Computerized Maintenance Management System................................. 100 Figure 49: CSO Closed-Circuit Television Dashboard with Indicators and Graphs for CCTV Inspections by Staff or Contractors...................................................................................... 101 Figure 50: Treatment Plant Maintenance Dashboard of Work Orders Closed by Shops..... 101 Figure 51 : Treatment Plant Maintenance Dashboard of Work Orders Backlog and Hours.. 102 Figure 52: Treatment Plant Maintenance Dashboard of Work Order Backlog by Shops Linked toCMMS ............................................................................................................................. 102 Figure 53: Work Order Types .............................................................................................. 105 10 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 246 of 461 Page 13 of 195 Figure 54: Cityworks Workflow Model ................................................................................. 108 Figure 55: Reliability Strategic Framework.......................................................................... 112 Figure 56: Central San's "Don't Just Fix It, Improve It" Maintenance Culture ...................... 115 Figure 57: Grease Gun Color Code Markings ..................................................................... 116 Figure 58: Failure Reporting Analysis and Corrective Action System (FRACAS) Workflow Diagram............................................................................................................................... 119 Figure 59: Daily Activities for Best Practice Framework ...................................................... 120 Figure 60: Before Optimization............................................................................................ 123 Figure 61 : After Optimization, Smooth Transition Eliminates Buildup Points....................... 123 Figure 62: Central San's Asset Condition Monitoring Program Begins with Uptime Elements ............................................................................................................................................ 126 Figure 63: Work-scheduling Process................................................................................... 127 Figure 64: Operations and Maintenance Staff are Now Involved in all Phases of the Asset Lifecycle .............................................................................................................................. 128 Figure 65: BIM Example: 3D Model of Cogen Gas Turbine Area Versus Actual Layout ..... 129 Figure 66: Performance and Monitoring Example: Bad Actors............................................ 134 Figure 67: Performance and Monitoring Example: Reactive Work Orders by Process........ 134 Figure 68: Performance Monitoring and Reporting Example: Regulatory Compliance, Safety, PM, and PdM Work Order KPIs........................................................................................... 134 Figure 69: Performance and Monitoring Example: Closed Work Order County by Shop .... 135 Figure 70: Performance and Monitoring Example: PMs and PdMs Resulting in Child Work Orders ................................................................................................................................. 135 Figure 71 : California Public Utilities Commission Fire Map ................................................. 137 Figure 72: Personnel Employment Circle ............................................................................ 140 Figure 73: Training and Development Checklists for Crafts ................................................ 141 Figure 74: Plant Information Management System Main Page ........................................... 141 Figure 75: Plant Maintenance Staff ..................................................................................... 142 Figure 76: Career Development Program............................................................................ 144 Figure 77: Lubrication Room ............................................................................................... 145 Figure 78: Monthly Internal Publication, "Lateral Connection" ............................................. 145 Figure 79: Mechanical Shop................................................................................................ 145 11 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 247 of 461 Page 14 of 195 Figure80: Tools Room ........................................................................................................ 145 Figure 81: Treatment Plant Maintenance Furnace Dashboard............................................ 150 Figure 82: Treatment Plant Maintenance Furnace Dashboard of Temperature and Sound Inspections .......................................................................................................................... 151 12 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 248 of 461 Page 15 of 195 Table 1: Initial Asset Management Program Steering Committee ......................................... 30 Table 2: Roles and Responsibilities....................................................................................... 31 Table 3: Central San's Asset Management Aligned with Strategic Goals.............................. 37 Table 4: Central San's Asset Management Stakeholders ..................................................... 39 Table 5: Total Spills by Year.................................................................................................. 47 Table 6: Asset Conditioning Scoring...................................................................................... 75 Table 7: Consequence of Failure Scoring ............................................................................. 75 Table 8: Condition Assessment Level with Aggregate Weight .............................................. 77 Table 9: Condition Assessment Level Weight Adjustments Based on Levels Performed...... 77 Table 10: Condition Assessment Weighting for Steam System............................................. 78 Table 11: Condition Assessment Weighting for Electrical Equipment ................................... 78 Table 12. Sewer Selection Criteria Categories...................................................................... 80 Table 13: Remaining Useful Life............................................................................................ 97 Table 14: Nominal Lifespans of Equipment........................................................................... 97 Table 15: Experienced Centered Maintenance (ECM) Analysis Results ............................. 121 Table 16: Procedure Update Owners .................................................................................. 132 Table 17: Computerized Maintenance Management System Reports................................. 132 Table 18: Workflow Monitoring Inboxes............................................................................... 133 Table 19: Contingency Options ........................................................................................... 136 Table 20: Job Classifications............................................................................................... 139 Table 21: Asset Management Plan Elements...................................................................... 148 Table 22: Asset Management Projects................................................................................ 149 Table 23: Asset Management Future Goals ........................................................................ 149 13 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 249 of 461 Page 16 of 195 �DOCUMENT VERSIONHISTORY Date Description1.0 December 2018 Initial Draft titled "Comprehensive Asset Management Dana Plan." Lawson 1.1 December 2022 Updated to include recent projects, overall re- Khae organization of topics, and new title: "Asset Bohan Management Plan." 1.2 April 2023 Reviewed by Central San Executives, Senior Engineers, Khae and staff. Updated to reflect comments and suggestions. Bohan Reviewed by Central San Engineering & Operations Committee on April 3, 2023. 1.3 December 2023 Updated to reflect comments, bigger graphics, and Khae added glossary. Presented to Central San Board of Bohan Directors on December 21, 2023. 14 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 250 of 461 Page 17 of 195 LIST OF • Software Usage Adobe ColdFusion Dynamic Reports, Spare Parts Management Tool Autodesk/innovyze InfoAsset Planner Risk Model Prioritized Capital Planning Autodesk/innovyze InfoWorks ICM Hydraulic Capacity Modeling Cityworks Computerized Maintenance Management System (CMMS) Esri ArcMap Software for Geographic Information System GIS Esri ArcPro Software for GIS Esri Dashboard Business Intelligence Analytics ITpipes Closed Circuit Television sewer inspection CCTV Laserfiche Document Management System Safe Software FME (Feature Manipulation Engine) Data Automation - ETL (Extract, Transform, and Load) Transdyn Dynac Plant Control System -Supervisory Control and Data Acquisition SCADA LIST OF ' • Abbreviati Meaning Abbreviation Meaning ACI Asset Condition Information Inst Instrument ACM Asset Condition Management ISO International Organization for Standardization AM Asset Management IT Information Technology AMIP Asset Management KPI Key Performance Indicators Implementation Plan AMP Asset Management Plan or LGIM Local Government Information Program Model A/N Aeration/Nitrification LOF Likelihood of Failure Aux Auxiliary LS Lift Station B&G or BG Buildings and Grounds MCC Motor Control Center BAAQMD Bay Area Air Quality MG Million Gallons Management District BACWA Bay Area Clean Water MGD Million Gallons per Day Agencies BCE Business Case Evaluation MI Maintenance Improvement BI Business Intelligence MRO Maintenance, Repair, and Operations BIM Building Information Modeling MRC Maintenance Reliability Center Board Board of Directors MSB Material Services Building BRE Business Risk Exposure Mech or MMB Mechanic Maintenance Building 15 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 251 of 461 Page 18 of 195 CA Condition Assessment NACWA National Association of Clean Water Agencies CAD Computer-Aided Design NPDES National Pollution Discharge and Elimination System CBM or CbM Condition-Based Maintenance O&M Operations and Maintenance CCCSD or Central Contra Costa Sanitary PdM Predictive Maintenance Central San District CCTV Closed-Circuit Television PDS Planning and Development Services acronym for Laserfiche CIB Capital Improvement Budget PG&E Pacific Gas & Electric CIP Capital Improvement Plan PIMS Plant Information Management System CM Corrective Maintenance PM Preventive Maintenance CMMS Computerized Maintenance PMO Planned Maintenance Management System Optimization COF Consequence of Failure POB Plant Operations Building CP Capital Projects POD Plant Operations Department CRL Certified Reliability Leader POF Probability of Failure CS Collection System PSPS Public Safety Power Shutoffs CSO Collection System Operations PVC Polyvinyl Chloride CSOD Collection System Operations QA/QC Quality Assurance/Quality Division Control CWEA California Water Environment RA Reliability Analysis Association CWMP Comprehensive Wastewater RAS Return Activated Sludge Master Plan DAF Dissolved Air Flotation RCA Root Cause Analysis DAFT Dissolved Air Flotation RCM Reliability Centered Thickener Maintenance DE Defect Elimination RDI Motion Amplification Technology company DJFI Don't Just Fix It, Improve It RE Reliability Engineering DP District Project ReW Recycle Water ECM Experience Centered RM Reactive Maintenance Maintenance Elec Electrical RTF Run to Failure EOC Emergency Operations Center RTI Risk Threshold Investigation EPA Environmental Protection RUL Remaining Useful Life Agency ERM Enterprise Risk Management SCADA Supervisory Control and Data Acquisition ERP Enterprise Resource Planning SOP Standard Operating Procedure ESST Emergency Sludge Storage SQE Safety, Quality, and Tank Effectiveness ETL Extract, Transform, and Load SQL Structured Query Language FE Final Effluent SSMP Sewer System Management Plan 16 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 252 of 461 Page 19 of 195 FME Feature Manipulation Engine Steam Project Steam and Aeration Blower Systems Renovation Project FMEA Failure Modes and Effects Sub Substation Analysis FRACAS Failure Reporting Analysis and SWGR Switchgear Corrective Action System FY Fiscal Year TP Treatment Plant GIS Geographic Information USA Underground Service Alert System GISP Geographic Information UE Ultrasound company System Professional HHW Household Hazardous Waste UW Utility Water HOB Headquarters Office Building UV Ultraviolet HR Human Resource W Water HVAC Heating, Ventilation, and Air WAS Waste Activated Sludge Conditioning ICM Integrated Catchment WDR Waste Discharge Requirements Modeling software 17 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 253 of 461 Page 20 of 195 EXECUTIVE . - This Asset Management Plan (AMP) describes the systems and strategic approaches that Central Contra Costa Sanitary District (Central San) utilizes to maximize value of assets. Asset Management is defined as the coordinated set of activities that engages all stakeholders to achieve organizational objectives from assets over their lifecycle. From the Board of Directors (Board) setting up an Asset Management Policy in 2014 to an Asset Management Implementation Plan in 2015 to creating analytics dashboards in 2022, the coordinated activities of various stakeholders have led to a solid Asset Management Program. Representative of our progress, Central San won Uptime Element's awards for Best Reliability Engineering Maintenance in 2018, Best Asset Management Program in 2020, and Best Work Execution Management Program in 2022. The plan describes the overall strategic approach to managing assets at Central San. Figure 1 describes Central San's Asset Management and Reliability Engineering journey throughout the years. Asset Management and Reliability Engineering Timeline - F bM Ilesi4•Ipn•#•i AssplKnowledge k Decision Hurnan S Capital O #Risk Management ' • ' Managernent # 1 ME I 1 ` Mana 1 1 1 Best Work ExecLdion ■ ■ k • - - Nogram Pefformance Management - Planning and Scheduling Operdtor Citywnrks .' Relrability BestRelliability Management ... Defect Maintenance Prograrn Con" Figure 1:Asset Management and Reliability Engineering Timeline Acronyms: • Asset Management (AM) • Computerized Maintenance Management System (CMMS) • Asset Management Implementation Plan (AMIP) • Maintenance,Repair,and Operations (MRO) 18 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 254 of 461 Page 21 of 195 Standards and Level of Service QUICK GLANCE AT THE ASSET MANAGEMENT A PLAN • Board Policy • Environmental Protection Agency (EPA) 10-Step Process Risks for Developing an Asset - „ 0 Risk Definition Management Plan0 Identifying the Risks • International Organization for - - • Quantifying the Risk Standardization (ISO) 55000 0 Condition Assessment series _ Program Decision Making Criteria: Drivers and Life • Compliance & Regulatory 0 Executive Level Risk Indices Cycle Management Requirements • Level of Service: Metrics from ASSET Drivers o Aging Infrastructure Strategic Plan MANAGEMENT PLAN o Regulatory Assets o Sustainability o Optimization • Asset Definition Capital Improvement Program • Sewer Gravity Main tea, CECEMBER2023 Asset Lifecycle • Sewer Force Main "` Remaining Useful Life • Pumping Stations Data Management and Utilization • Treatment Plant Equipment and Piping for Decision-Making • Recycled Water Distribution • General Improvements Asset Management Information Systems and Tools • GeoPortal - Dynamic web- Asset Management Approaches based map and data access • Single Asset Registry General Approaches • Asset Hierarchies o Operations Staff Development • CMMS o Maintenance • Closed-Circuit Television (CCTV) • Specific Approaches 0 Personnel Advancement Inspection System Automatic o Collection System 0 Educational Resources p ram o Treatment Plant • Internship Pro Vehicle Locating g • Underground Service Alert (USA) o Recycled Water 0 Mentorship Program Locate Tickets • Standard Operating Procedures (SOP) 0 Externship Program • Spare Parts Management Monitoring and Managing System 0 Management and Supervisory Academy Application Breakdowns y • Reliability Engineering Report Updating and Maintaining Procedures Career Development Program• Document Management g Performance Monitoring and Reporting Performance Recognition, System Operational Resilience and Disaster Appreciation, and Beyond • Risk and Flow Modeling Preparedness 0 Conferences and Training Software • Team Accomplishments 19 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 255 of 461 MANAGEMENT ROADMAP ASSET Cominue Asset do 11% 2023 '. ' :I I.roue migration . -I - Geometric — { N er ork to UuMy 2023 Nerwor k — Continue Comprehensive Asser' Evaluation for I the Prioritization s of � � � � � � 0 23 Vemcal Assets - E — %0 + * r CCI I.- I ILI& the De--.-elapment of W* Milo Vw Data AnafyLics 2024 mop Digiuze Operator Round Sheets orb E — am 2025 Complete she Asset Registry tor Treatment Plant Pipin-g 2025 Underground Utility Smart U"Lility' Complete � % � S4andarOzation Gf ` - Smart MCCs 02 Across the District iww _ Smart Utility. Pilot +r . Asset He.3I,,h - # Monitoring 2027 Technology Smart Utility; Asset i Handoff Tracking Monitoring, Apply Ai/Machine Learning 2028 Where Applicable f Dynamic Asse* J* Managenien,- Plan Page 23 of 195 BACKGROUND 1 . INTRODUCTION Established in 1946, Central San is located about 30 miles northeast of San Francisco and provides wastewater services for nearly 500,000 residents and more than 15,000 businesses in central Contra Costa County. This service territory covers 146 square miles and includes California Alamo, Clyde, Danville, Lafayette, Martinez, Moraga, Orinda, Nevada Pacheco, Pleasant Hill, San Ramon, Walnut Creek, and unincorporated areas within central Contra Costa County. Central San also treats wastewater from an additional 37 square miles for residents of Concord and Clayton under a 1974 contract with the City of Concord. 4 Central San Service Area Figure 3:Central San in Relation to California and Nevada o�`f _. ®�Fhw�,art F.lrl�d 9 • p --. Pleaaail-t Co [7,5r1 H lII - ,� TiJa Ytan wa i C leek n.`oin�Y`. e,._p�nf La faY ette t n$va u 3, 1 E� ■ ua91....an n�.�� ■n a a Figure 4:Headquarters,Treatment Plant,and HHW Collection Facility 1\ No.,.,aoJM o.a9a , - ■ ndm'ni r an(Open ns Facill:y 5erverlor�Malrv_j>�°:� San R mo c- n Interrenror5-Main l.2t inlnk5ewzr Mzln jt]2"A52G Oalfall � ycled woln Noin NneYcled Nolnm ■R O wt slcuvicr Called _ _aLn dNo,=hodHrt ;doe\Vr¢,e l[eitlonent A k'nsavmtcr Trca[mcut and ll....Mid p l nrtin�a,r,eol:wane c4uerno 4 Figure 2:Central San Service Area by the Numbers Figure 5:Collection System Operations Division Buildings 21 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 257 of 461 Page 24 of 195 • Serves nearly 500,000 residents and more than 15,000 businesses in central Contra Costa County =� • Maintains over 1,500 miles of sewer pipelines and 18 pumping stations to carry wastewater to Central San's regional treatment plant in Martinez 4 Treats more than 13 billion gallons of wastewater per year i • Produces more than 500 million gallons of recycled water every year for irrigation and industrial uses • Collects over 2 million pounds of household hazardous �- waste (HHW) per year and strives to reuse or recycle about 89% of those materials • Collects more than 4,000 pounds of unwanted medications per year Figure b:Cleaning • Achieved an average 4.0 out of 4.0 customer satisfaction rating for sewer emergency response (as of Quarter 2 in Fiscal Year (FY) 2022-23) • Serves more than 20,000 students through school education programs every year • Welcomes more than 500 participants to treatment plant tours and speakers bureau presentations per year 2. VISION AND MISSION Central San's vision is to be an innovative industry leader in environmental stewardship and sustainability, while delivering exceptional service at responsible rates. Our mission is to protect public health and the environment. A. ORGANIZATIONAL STRUCTURE Central San is governed by a Board whose five members are elected by voters to serve a four- year term. The staff is managed by a General Manager who serves as the Chief Operating Officer, implements the Board's policies, and oversees the business of Central San. The organization is divided into three departments- Finance and Administration, Operations, and Engineering and Technical Services-each overseen by a Director or a Deputy General Manager. Central San's main headquarters, Board Room, treatment plant, and HHW Collection Facility are located at 5019 Imhoff Place in Martinez. Central San's Collection System Operations (CSO) headquarters is at 1250 Springbrook Road in Walnut Creek. 22 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 258 of 461 Page of • Electorate Figure 7:Central San Board of Directors Secretary of General Counsel for the District Manager the District Deputy General HR 8 Organizational Deputy General Internal Manager of Development Manager of Engineering Auditor Administration Manager & Operations Finance Human (including Capital Purchasing) Resources Projects Information Safety Environmental 8 Technology Regulatory Compliance Communications �` Planning 8 Intergovernment Development al Relations Services Risk Resource Management Recovery Plant Operations Plant Maintenance Optimization Collection System Operations Organization Structure Page 26 of 195 B. HISTORY In the 1940s, central Contra Costa County was a rural area of farms, orchards, and a few small towns. With the end of World War II, a building boom began. As the nearby cities of San Francisco, Oakland, and Berkeley grew, so did the population of Contra Costa County. r- The population growth ultimately resulted in a sanitation crisis, with most of the county depending on septic systems, often inefficient due to the area's heavy adobe clay soil. At the ` time, state health authorities considered the ' x polluted conditions arising from those septic tanks to be among the worst in California. With septic tanks overflowing and waterborne diseases such as typhoid becoming a potential threat, civic leaders rallied public support for a solution. An election held on June 24, 1946, approved a proposal to form a sanitary district for central Contra Costa County areas. On July 15, 1946, the County Board of Supervisors passed a resolution officially creating the Central Contra Costa Sanitary District. Work began immediately and within 26 months, Central San's newly constructed main sewer trunk line and treatment plant were operational. At that time, Central San's service area population was 15,000; the treatment plant's capacity was 4.5 million gallons per day (MGD), and the collection system consisted of 50 miles of sewer pipe. That original system was expected to handle the area's wastewater for at least two decades. But by 1952, it was apparent that demand would soon exceed capacity. Central San began a series of expansions and improvements to meet the needs of a rapidly growing population -activities that have continued throughout the Central San's history. C. COMMUNITY PROFILE For nearly 77 years, Central San has been proud to serve its customers in the San Francisco Bay Area's central Contra Costa County region. The service area is at the foot of Mount Diablo (3,848 feet), whose state park and foothills offer hiking trails and open space preserves that the neighboring residents frequently use. The cities served by Central San are also some of the most historic in California. Martinez, where the headquarters is, was a key crossing point over the Carquinez Strait for the Pony Express. Its downtown is notable for its preserved historic buildings, including the John Muir National Historic Site. A short distance away, the Cities of Concord, Walnut Creek, and San Ramon boast revitalized shopping districts, drawing retailers and restaurants from other parts of the state to open locations there. One of Central San's largest customers, the City of Concord, is working on converting a former Naval Weapons Station into a Community Reuse Project, which will include parks, housing, office, retail, and the restoration of Mt. Diablo Creek. Central San is proud to be part of the effort to make the project as sustainable as possible by supplying recycled water for irrigation in this development. 24 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 260 of 461 Page 27 of 195 In recent years, the service area's population has boomed, partially due to its accessibility to San Francisco and Silicon Valley via public transit. Most of the people of Contra Costa County live along the busy 1-680 corridor that connects the ` North Bay to Silicon Valley. The area in which tx Central San operates is a growing community to whom this agency is honored to provide its core , services, including educational messaging to instill } { the environmental values foundational to Central i{,• . r�;,, = y. r.. _ San as an organization. Enormous changes have occurred since Central San's beginnings in 1946 as a small agency serving ' a rural area: the size, population, and z_. characteristics of the service area; the processes . and technologies used to treat and collectkc'• wastewater; the environmental awareness; and stringer air and water quality standards and regulations. Our unwavering dedication and commitment to protecting public health and the environment have not changed. 3. SERVICES OVERVIEW Since 1946, Central San has provided safe and reliable wastewater collection and treatment for central Contra Costa County residents. Our services include a complex treatment plant, collection system, 18 pumping stations, recycled water facility, HHW collection facility, and a sophisticated water quality laboratory. A. COLLECTION SYSTEM The collection system consists of three main components: pumping stations, gravity sewers, and force mains. The Collection System Operations Division (CSOD) site is strategically located at 1250 Springbrook Road in Walnut Creek to provide staff centralized access to maintain the collection system. The campus consists of an office building, warehouse, fleet shop, materials storage structures, emergency generator, fueling system, and parking facilities. B. WASTEWATER TREATMENT The treatment plant was expanded after the Clean Water Act was passed, with most facilities built in the late 1970s. Central San's National Pollution Discharge and Elimination System (NPDES) permit allows discharge up to 53.8 MGD on an average dry-weather basis and lists a 250 MGD peak wet weather design flow. Due to hydraulic limitations within the treatment plant, the treatment plant operates in secondary bypass mode when influent flows exceed approximately 90 MGD, and in raw influent bypass mode when influent flows exceed approximately 175 MGD (Central San, 2019). Central San's treatment plant is currently treating approximately an average of 30.9 MGD on a dry-weather basis. Central San is one of the last agencies on the West Coast with sewage sludge incinerators. In addition to the communities in the sewer collection service area, Central San provides wastewater treatment for the City of Concord and the Town of Clayton, which own and operate 25 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 261 of 461 Page 28 of 195 their collection systems that convey wastewater to Central San's interceptor system and wastewater treatment plant. C. RECYCLED WATER After wastewater is treated and disinfected at the treatment plant, most is discharged into Suisun Bay, but a portion receives further treatment at the Recycled Water (ReW) Filter Plant. Central San produces approximately 600 million gallons (MG) of recycled water annually. Of that, the treatment plant uses about 400 MG per year. Central San pipes the remaining recycled water to Martinez, Pleasant Hill, and Concord locations for some dual plumbing installations but primarily for landscape irrigation, including golf courses, parks, and sports fields. The treatment facilities were constructed in the mid-1970s as part of the wastewater treatment plant expansion. Central San opened a Residential ReW Fill Station in 2016 at the HHW Collection Facility in Martinez. The station provides free recycled water (300 gallons maximum per trip) for hand- watering lawns, gardens, and landscaping to residential customers of Central San, Mt. View Sanitary District, and the Cities of Clayton and Concord. For 2021, the facility provided an annual of 5.1 MG by residential fill and truck fill of 5.5 MG. Central San's Hydrant Truck Fill Program provides recycled water to licensed contractors for uses such as dust control, soil compaction, landscape irrigation, and sewer flushing. D. HOUSEHOLD HAZARDOUS WASTE COLLECTION FACILITY Central San opened a HHW Collection Facility at 4797 Imhoff Place in Martinez in 1997 to reduce and prevent pollution through reuse, recycling, and proper disposal of hazardous waste (motor oil, paint, pesticides, cleansers, batteries, etc.). Residents within the HHW service area drop off their items without charge or appointments. Small businesses are charged a nominal fee and require an appointment. The facility does not serve large businesses. The HHW service area is slightly larger than the Central San wastewater collection and treatment service area. It includes all of San Ramon and the Mt. View Sanitary District service area. Since it opened, the facility has collected more than 47 million pounds of HHW1. The facility currently collects about 2 million pounds each year, 89%of which is reused, recycled, or used as a beneficial fuel. The number of households using the HHW program grew by more than 14%over the past 20 years. Program participation has also increased steadily from the 7.8% of households served during the opening year in 1997-98 to nearly 17%in FY 2016-17. E. PHARMACEUTICAL COLLECTION Central San operates a Pharmaceutical Collection Program in partnership with local law enforcement agencies to provide free, safe disposal of unwanted medications. The program offers 13 collection sites throughout the Central San service area that safely dispose of more than 16,000 pounds of pharmaceuticals each year. Staff educates customers not to flush drugs Central San, "25 Years of Household Hazardous Waste Collection",Pipeline,Fall 2022, hftps://www.centralsan.org/sites/main/files/file-attachments/pipeline fa112022.ndf?166873231 1 26 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 262 of 461 Page 29 of 195 because they often contain chemical compounds that are not entirely removed during treatment and can pollute the bay. 27 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 263 of 461 Page 30 of 195 ORGANIZATIONALASSET MANAGEMENT 1 . PURPOSE The purpose of this Asset Management Plan (AMP) is to document the systems and process of the Asset Management Program, the current state of Central San's assets, and the strategic and dynamic implementation throughout the life cycle of assets. This plan is a primary example of Central San's commitment to the environment, innovation, optimization, and continuous improvement. The purpose of the AMP is to document the following: • An overview of Central San's assets, summarizing the current condition of assets and their overall performance • System operation and maintenance strategies and documented procedures • Level of service standards, including performance indicators and adopted targets • Proposed actions to implement the above operation, maintenance, and strategies • Stakeholder engagement and communication plan A. WHAT IS ASSET MANAGEMENT? Central San relies on responsible and robust maintenance practices to help maintain equipment and facilities. Asset Management is a coordinated set of activities that engages all stakeholders to achieve organizational objectives from assets over their lifecycle. Asset Management is the systems and processes for maintaining and replacing these assets, such as manholes, cleanouts, pipes, pumps, buildings, tanks, and process equipment. Asset Management improves Central San's ability to make the right decisions, at the right time, at the right cost, and for the right reason by having the appropriate data and information available. According to U.S. Environmental Protection Agency (EPA): "Asset management is a process water and wastewater utilities can use to make sure that planned maintenance can be conducted and capital assets (pumps, motors, pipes, etc.) can be repaired, replaced, or upgraded on time and that there is enough money to pay for it. Asset management is the practice of managing infrastructure capital assets to minimize the total cost of owning and operating these assets while delivering the desired service levels. Many utilities use asset management to pursue and achieve sustainable infrastructure. A high- performing asset management program includes detailed asset inventories, operation and maintenance tasks, and long-range financial planning. Each utility is responsible for making sure that its system stays in good working order, regardless of the age of its components or the availability of additional funds. Asset management programs with good data—including asset attributes (e.g., age, condition, and criticality), life- cycle costing, proactive operations and maintenance (O&M), and capital replacement plans based on cost-benefit analyses—can be the most efficient method of meeting this challenge" (EPA, 2021). According to International Organization for Standardization (ISO) 55000, Asset Management is the "coordinated set of activities of an organization to realize value from assets." To deliver safe, reliable, and high-quality services in a sustainable manner, Central San manages a significant amount of infrastructure with thousands of mechanical and electrical equipment, structures, instruments, and other assets. Central San's robust maintenance practices have 28 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 264 of 461 Page 31 of 195 extended the lives of many assets. From the 1970s to the 1990s, Central San's maintenance needs were relatively simple. Plant Maintenance staff employed a simple computer program for Preventative Maintenance (PM) work orders and 3-part carbon-copy work order slips for correctives. At this time, "firefighting" maintenance took a significant portion of each week and did very little proactive maintenance work. Near the end of this era, the Plant Maintenance Division began its first attempts at vibration and oil analysis programs. In the late 1990s, Central San transitioned to using full-fledged Computerized Maintenance Management System (CMMS) programs: Mainsaver for the Treatment Plant and Sussex (later Accela) for the Collection System group. At this time, the plant's assets were inventoried. By the early 2000s, Central San had a comprehensive PM program expanded to cover nearly all assets.This program served well for many years, but by 2014, new challenges faced by the organization precipitated a move to modern practices. These challenges included: • The sheer volume of assets in the treatment plant, plus undocumented underground assets, and lacking integrated knowledge of vertical assets • Multiple concurrent initiatives requiring a modern maintenance program in their implementation (e.g., CMMS, Reliability Centered Maintenance (RCM), Criticality matrix, fault action codes, condition assessments, Enterprise Resource Planning (ERP), and 20-year Master Plan development) • Day-to-day operational and maintenance needs under limited staffing and stricter regulations • Retirement of staff with significant institutional knowledge, often with no one else in that position (no redundancy) • Succession planning and staff development time requirements • Changes in regulatory requirements necessitate additional PM tasks and accelerated replacement • Learning curves for new techniques in Condition-Based Maintenance (CbM) and Predictive Maintenance (PdM) • Decreases in operating performance (e.g., pump efficiency, losses in steam, and aeration systems impact to redundancy at peak loads) • Challenges of creating new asset registry entries and new PMs for the new equipment installed during capital projects B. BOARD POLICY The Central San Board adopted the Asset Management Policy on November 20, 2014. The policy recognizes that Central San has invested significant resources in its assets and aims to optimize its lifecycle to deliver high-quality and reliable services sustainably for customers with an acceptable level of risk (for policy see Appendix 1. Board Policy 015, Asset Management). An asset is a physical or software asset with a replacement value greater than $5,0002 and useful life of at least two years, may have a maintenance program, or is critical to operations, regulatory, or safety requirements, or is controlled by the Treatment Plant or Pumping Station Supervisory Control and Data Acquisition (SCADA) systems. The framework includes these directives: • Data and Knowledge • Information Systems 2 The capitalization threshold is specified prospectively in the annual budget document and retrospectively in the Annual Comprehensive Financial Report.The$5,000 limit is currently under review. 29 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 265 of 461 Page 32 of 195 • Contracting • Processes and Practices • People • Organization C. ASSET MANAGEMENT IMPLEMENTATION PLAN (2015) The objectives of the plan are to: • Demonstrate Central San's approach to optimizing the lifecycle management of the asset portfolio • Describe how asset management will deliver the organizational objectives to implement and comply with the Asset Management Policy • Describe initiatives and plans of asset-level activities for the next five years With a renewed commitment from the Board in 2014, an Asset Management Policy and Implementation Plan were developed. Central San began creating a comprehensive Asset Management Plan with documented processes and procedures for data management, analysis, and evaluation for the treatment plant, collection system, pumping stations, recycled water distribution system, and general improvement assets. Central San established a strategic framework for improving asset maintenance efficiency and functional reliability. This framework consisted of the following goals: • Systematic development of a comprehensive maintenance program based on asset criticality and consequence of failure (COF) • Identification of the most effective PM tasks for treatment plant equipment and systems • Integration with a CMMS to establish a repeatable program with documented processes and procedures • Implementation of Azteca's Cityworks Asset Management System platform; Cityworks replaced two existing platforms, Accela and Mainsaver • Implementation of the Reliability Centered Maintenance (RCM) Program By improving and optimizing asset management processes, Central San will gain knowledge of the assets owned, the remaining useful life (RUL) to manage, the amount of investment required, and the business risk it faces to promote sustainable management practices. 2. GOVERNANCE The Asset Management Program requires collaboration amongst the Divisions of the organization to create a cohesive program. The success of the program requires support from executive leadership. The initial Asset Management Program governance started in June 2015 with an Asset Management Steering Committee. This committee comprised representatives from each of the three Divisions within the Engineering and Operations Divisions and one representative from the Administration Department as listed in Table 1. Table 1: Initial Asset Management Program Steering Committee Department Administration One from any Division Engineering &Technical Services Capital Projects 30 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 266 of 461 Page 33 of 195 Planning and Development Services Environmental and Regulatory Compliance Operations Collection System Operations Plant Operations Plant Maintenance The committee worked on replacing the CMMS, replacing the outdated web-mapping system, contributed to the Comprehensive Wastewater Master Plan (CWMP) as part of the condition assessment task, and implemented the steps outlined by the Asset Management Implementation Plan. The Board policy states that Central San will establish organizational responsibility to lead and manage a comprehensive Asset Management Program. Currently, the Optimization Division leads this effort with the support of Central San leadership across Engineering and Technical Services, Operations, Finance, and Administration. Staff has leadership support, as shown in Table 2, and various stakeholder interests in Table 4: Central San's Asset Management Stakeholders. A. EXECUTIVE LEADERSHIP SUPPORT Table 2: Roles and Responsibilities • - Responsibilities • General Manager Provide visible leadership and commitment to the Asset Management Program • Alignment with Strategic Goals • Director, Engineering and Technical 0 Provide visible leadership and Services commitment to the Asset Management • Deputy General Manager, Operations Program • Deputy General Manager, Finance 0 Maintain frequent communications with and Administration Asset Management Division leaders • Monitor and review asset management- related progress and performance • Empower staff to guide the Asset Management Program, coordinate participation among staff, and resolve or elevate asset management-related conflicts • Communicate any concerns regarding the Asset Management Program to staff • Division Managers Provide visible leadership and • Senior Engineers commitment to the Asset Management • Plant Operations Department Program Managers Coordinate and communicate with • Superintendents departmental asset management • Asset Management Program champions to enable and ensure the Administrator adoption of asset management principles, practices, processes, and tools • Identify needed asset management training and continued professional 31 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 267 of 461 Page 34 of 195 development, provide training, and improve staff competencies in asset management • Lead the engagement of departmental staff to ensure the adoption and operation alization of asset management principles, processes, and practices within the department • Optimize maintenance by using the appropriate maintenance analysis methods (e.g., planned maintenance optimization, failure mode and effects analysis, and reliability centered maintenance) • Manage and administer asset management applications, software, and the accuracy of asset registry. Identify asset data gaps and work with department directors to gather the information • Manage asset handover for improvements or new construction • Track and report results against asset management-related performance goals • Attend and participate in the Cityworks User Group • Ensure that department directors and staff are appropriately engaged in the Asset Management Program, its processes, and practices • Information Technology (IT) Align with Technology Master Plan • Support for software, server, and database infrastructure 3. COMPLIANCE AND REGULATORY REQUIREMENTS Projects must reliably comply with regulatory requirements designed to protect human health and the environment, including planning to anticipate future regulatory requirements. Potential regulatory issues include biosolids management, nutrient limits, air emissions, and constituents of emerging concern. A. COLLECTION SYSTEM The regulatory driver for the collection system is the State Water Resources Control Board Statewide Sanitary Sewer Systems General Order, adopted in 2006 and on December 6, 2022 (2022-0103-DWQ (Division of Water Quality) effective June 5, 2023. The regulations address aging infrastructure and expanded the previous order to include privately owned systems (California State Water Resources Control Board, 2023). ORDER 2022-0103-DWQ can be found online. https://www.waterboards.ca.gov/board decisions/adopted orders/water quality/2022/wqo 202 2-0103-dwa.pdf 32 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 268 of 461 Page 35 of 195 CONTEXTOF ORGANIZATION 1 . LEVELS OF SERVICE A. STRATEGIC PLAN SUMMARY Central San develops its Strategic Plans on a two-year cycle while tracking progress quarterly and producing performance reports annually. The Strategic Plans establish policy direction, outline core commitments, focus initiatives, and track performance with key metrics. The development of the Strategic Plan begins with the Board's re-adoption of Central San's Mission and adoption of the revised Vision, Values, and Goals in December every two years. These updates emphasized the agency's commitment to the environment, innovation, optimization, and continuous improvement. Staff holds a workshop the following month to update the Strategic Plan to reflect current priorities. The Strategic Plan contains five components: Goals, Strategies, Initiatives, Key Success Measures, and Metrics. The strategies outline Central San's approach to achieving its goals, overcoming challenges, accomplishing its mission, and meeting the community's needs efficiently and effectively. The Initiatives describe the actions staff will take, and the Key Success Measures delineate the tasks to fulfill those Initiatives. The Key Metrics set targets, track progress, and evaluate performance. Starting with the development of the FYs 2016-18 Strategic Plan, staff has used the Effective Utility Management model as a tool to identify practices and procedures to improve operations and move toward continued sustainability. The Effective Utility Management framework was initially developed in 2007 by the American Water Works Association, U.S. EPA, and nine other association partners representing the U.S. water and wastewater sector. It consists of 10 attributes that provide concise focus areas for effectively managed utilities and what they should strive to achieve. 33 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 269 of 461 Page 36 of 195 *' Stakeholder Understanding Product and Support Q ua lily Water Re source cu sto m e( � Sustaanabilitp Satisfaction Effective Employee and Comm uni#y Leadership Sustairtiability Management1 Development Enterprise Operational Resiliency Optimization rt Infrastructure Filrtiian6ai Stra#egg+and ViabiNty Performance 1 � Figure 8:Effective Utility Management Framework 34 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 270 of 461 Page 37 of 195 I' Product Quality- Produces "fit for purpose"water that meets or exceeds full compliance with regulatory and reliability requirements and is consistent with customer, public health, ecological, and economic needs. t rf Customer Satisfaction - Provides reliable, responsive, and affordable w services in line with explicit, customer-derived service levels. Employee and Leadership Development - Recruits and retains a workforce that is competent, motivated, adaptive, and safety focused. Operational Optimization - Ensures ongoing, timely, cost-effective, reliable, and sustainable performance improvements in all facets of its operations in service to public health and environmental protection. r Financial Viability- Understands the full life-cycle cost of utility operations and the value of water resources. Infrastructure Strategy and Performance - Understands the condition of and costs associated with critical infrastructure assets. _e Enterprise Resiliency - Ensures utility leadership and staff work together internally, and with external partners, to anticipate, respond to, and avoid problems. Community Sustainability-Takes an active leadership role in promoting and organizing community sustainability improvements through collaboration with local partners. Water Resource Sustainability- Ensures the availability and sustainable 4management of water for its community and watershed, including water resource recovery. Stakeholder Understanding and Support- Engenders understanding and support from stakeholders, including customers, oversight bodies, i. community and watershed interests, and regulatory bodies for service levels, rate structures, operating budgets, capital improvement programs, and risk management decisions. Figure 9:Effective Utility Management Attributes 35 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 271 of 461 Page 38 of 195 The metrics in the current Strategic Plan that relate to assets and customer expectations serve as Central San's Levels of Service. Levels of Service are specific, measurable parameters that reflect what customers deem important and expect from their utility. -= i The metrics are assigned to process owners, typicallyr members of the Management Team, who then delegate the 7 i iii�pp+nnrru d^lq� Rrn11R��� j Pi i I _L� 1 III{IIIiI Mr' i II;llrllrr �19 Ri-, ill Ei ilrlEn&INNAl data collection, analysis, and monitoring to subordinate i�I" ��I f staff. ' The performance for each metric is updated quarterly by staff and then reported to the General Manager and the Board's Administration Committee. At the beginning of each fiscal year, a report of the previous --- n. fiscal years achievements toward the Strategic Plan is • • presented to the Administration Committee and full Board. To • • - enviro - - Please see Central San's website, https://www.centralsan.org/post/strategic-plans, for the OUR • -- current and previous two-year Strategic Plans and Annual To be an innovative industry leader in Reports. environmental stewardship d 2. STRATEGIC ALIGNMENT sustainab ility,while delivering exceptional service at responsible rates A. ALIGNMENT WITH MISSION, VISION, AND VALUES OUR VALUES Central San is focused on providing high-quality services at •_cisions' - • - ' ' ' •• ily fair and affordable rates to its customers. Central San has and how we fulfill our long emphasized optimizations to realize our vision of being mission, vision, and goals an industry-leading organization in innovation. Our Strategic Customer Goal of Innovation and Optimization is dedicated to this Integrityp __ effort. Management nurtures the ingenuity of our staff and Innovation encourages interdepartmental teamwork, leveraging of Environmental Sustainability data, and the use of new technologies to help us find new Diversity, ways to optimize our work while maintaining our high level of customer service. Central San runs an internal Innovations competition and in 2020, the Asset Management workgroup Figure 10:Central San Mission,Vision,and Values received the "Most Efficient" award for the CSO Cleaning Optimizing application and were featured in 2022 for their Ultraviolet (UV) Resource Planning Dashboard. Staff actively monitor performance data via software databases and curated dashboards created in-house. Staff use these findings to adjust and employ system-wide optimizations. This is to ensure staff operate efficiently and effectively, making strategic business decisions based on accurate real-time data. Central San uses asset management principles for a holistic approach for critical decisions on infrastructure project timing, prioritization, and rehabilitation versus replacement. This approach ensures vital equipment reliability and operational availability. In service to our customers, our Strategic Goal of Infrastructure Reliability solidifies our commitment to optimize lifecycle costs, maximize asset value, and manage risk when investing in and maintaining our assets. Our Asset 36 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 272 of 461 Page 39 of 195 Management Program uses data, risk scoring, triple bottom line, decision matrices, accountability, and financial modeling to provide visibility into the lifespan and costs of our critical assets. Our Comprehensive Asset Evaluation process expands our Condition Assessment (Planning and Initial Evaluation, Inspection, and Engineering Analysis) efforts to systematically factor in additional drivers (capacity, regulatory, and sustainability/optimization) in assigning RUL for critical assets. Business case evaluations are performed on major projects, using a criticality scoring system for Probability of Failure (POF), Consequence of Failure (COF), and Business Risk Exposure (BRE). Along with analysis of maintenance and asset condition data, and hydraulic model output, staff created a 100-year sewer rehabilitation and replacement forecast and program. The robust planning, capital improvements, operations, and maintenance procedures have produced record-low spills. B. ALIGNMENT WITH STRATEGIC PLAN In alignment with Central San's Strategic Plan, staff defined the objective of our Asset Management Program to be the following: "using technology to optimize the lifecycle of Central San's assets and deliver reliable, high-quality services in a sustainable manner." Table 3 illustrates this plan's alignment with Central San's Strategic Plan. Table 3:Central San's Asset Management Aligned with Strategic Goals Goal Asset Management Plan and Reliability Alignment Goal 1 -CUSTOMER Reduce and minimize impacts to residents and businesses by improving AND COMMUNITY: and optimizing asset management processes. Participate in citizens Provide exceptional Central San Academy. Provide leadership roles in industrial organizations service such as California Water Environment Association (CWEA), Bay Area Clean Water Agencies (BACWA), Association of Asset Management Professionals, and Reliability Leadership Institute. Goal 2- Recently optimized asset registry and CMMS process to add regulatory ENVIRONMENTAL fields to asset registry to improve work orders with regulatory assets. STEWARDSHIP: Meet Maintain and update the recycled water asset registry. Developed regulatory application tools to optimize CSO's planning of cleaning schedule. requirements, promote Achieve 100%compliance in all regulations, continue to attain National sustainability, and Association of Clean Water Agencies (NACWA) performance, perform identify and reduce condition assessment of key assets such as Outfall and Steam, and contributions to climate Aeration system. Reduce natural gas usage through energy efficient change and mitigate its improvements. Track energy usage and explore energy efficient ways to impacts improve such as lighting, solar, and heating, ventilation, and air conditioning. Continue to monitor key performance indicators (KPI's) for regulatory compliance. Goal 3-WORKFORCE Proactively plan for future operational staffing needs. Foster relationships DIVERSITY & across all levels of Central San. Inspire employee engagement. Develop DEVELOPMENT: Recruit, training videos for CMMS and engage stakeholders. Develop tools and educate, empower, applications to empower staff. Continue developing Standard Operating and retain a workforce Procedures (SOPS) and cross-training amongst workgroups. Provide training from diverse in different modalities. Attain certifications such as Geographic backgrounds Information System Professional certification (GISP) or Certified Reliability Leader (CRL). Goal 4-GOVERNANCE Risk-based planning of infrastructure investments optimizes spending and & FISCAL reduces costly asset failures. Integrate data from the CWMP and Asset RESPONSIBILITY: Uphold 37 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 273 of 461 Page 40 of 195 integrity, transparency, Management Program into analyzing long-term capital improvement and wise financial needs. Perform asset valuation. management in an effective governing model Goal 5-SAFETY & Provide safety training and safety gear for fieldwork, participate in Safety, SECURITY: Provide a participate and engage in Emergency Operations Center (EOC) safe, secure, and Activation Exercises, participate in the Administration and Engineering healthful workplace Safety Team. that foresees and addresses threats Goal 6- Manage assets optimally to prolong their useful life by expanding reliability- INFRASTRUCTURE based approaches to managing assets, prioritizing asset lifecycle for RELIABILITY: Maintain rehabilitation and replacement, PM, and inspection. Execute long-term facilities and equipment capital renewal and replacement program. Perform condition assessments to be dependable, on critical equipment and incorporate into Capital Improvement Plan (CIP) resilient, and long- or maintenance program as appropriate. Protect personnel and assets lasting from threats and emergencies. Improve Asset Condition Monitoring (ACM) program, add service contracts to CMMS, KPIs of Shop Metrics such as PM, Safety and Regulatory compliances. Goal 7-INNOVATION Improve and modernize operations through technology and efficiency AND AGILITY: Optimize measures. Maintain a safe working environment for employees and the operations for public during the pandemic. Plan various scenarios such as Public Safety continuous Power Shutoffs (PSPS), cross train staff and implement succession planning improvement, and and staff advancement. remain flexible and adaptable C. ALIGNMENT WITH FINANCIAL PLAN (BUDGET), CAPITAL IMPROVEMENT PLAN, AND TECHNOLOGY MASTER PLAN To provide high-quality, affordable services to its customers at fair and reasonable rates, staff took a prudent and proactive approach to financial management and long-range planning, reflected in financial Board Policies and Strategic Goal of Governance and Fiscal Responsibility. To understand the total cost of providing the service of operating a complex system, staff conduct annual financial workshops, budget adoptions, and updates of the Ten-Year CIP. Through prudent financial planning and spending, Central San demonstrates fiscal responsibility to ensure that the rate structure will support ongoing and future needs to maintain high-quality customer service, comply with emerging regulations, and renew and replace aging infrastructure. The condition assessment of assets aims to validate and inform the prioritization of capital improvements. Central San uses video inspection and feedback from cleaning crews to evaluate the condition of its 1,500-mile network. Data analysis is conducted by Engineers to develop the CIP and the Capital Improvement Budget (CIB). Central San uses a risk-based analysis to develop sewer rehabilitation and replacement rates, and budgetary estimates for preparing the 10-year CIP. Central San uses risk-based asset management software that leverages existing Geographic Information System (GIS) and CMMS data, hydraulic model output data from Innovyze's InfoWorks hydrodynamic model, CCTV, and sewer cleaning data. 38 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 274 of 461 Page 41 of 195 The Technology Master Plan is in progress, and the Asset Management workgroup will work with IT staff and consultants to develop a technology roadmap for GIS and Asset Management. 3. STAKEHOLDER ENGAGEMENT AND COMMUNICATION PLAN A. STAKEHOLDER ENGAGEMENT Engagement with stakeholders is key to a successful asset management implementation and program. Effectively engaging and communicating with Central San's stakeholders is a priority and an essential element within ISO 55001. In collaboration with appropriate stakeholders, staff developed procedures, workflows, criteria, asset information, and various applications to help staff be more proficient in their jobs. Below is a list of how staff engage with each department and its divisions. B. INTERNAL STAKEHOLDERS Table 4:Central San's Asset Management Stakeholders Department Examples of Stakeholders Engagement Office of the • Sets the policies for the Asset Management Plan General • Receive asset management activities and updates Manager 0 Human Resource (HR) provides info for Cityworks employee labor costs • Internal audit conducts performance and internal control audits in various areas including related to Asset Management and the overall objective of safeguarding assets Office of the 0 Requests service boundary for County Board of election, public data Secretary of the request, and record documents District 0 Request boundary by District elections and web map application to find divisions based on addresses Finance & a IT provides support for software acquisition and licensing, infrastructure Administration support for servers and databases Department Engage with Material Services to develop Spare Parts Application for Operations • Our website shows construction project-specific information such as descriptions, maps, and contact info, the project-specific interactive web maps show mains with construction method and locations of the construction • Risk Management holds an EOC exercise, where GIS and Asset Management services are requested to provide asset and maintenance history for risk-related requests • Risk Management uses Cityworks to manage its security assets, such as video cameras, card readers, and alarms • Finance works with the Asset Management group to advance the interface and tracking of assets for financial reporting, security, and asset management purposes Engineering & 9 GIS/Asset Management workgroup provides support and Technical development of Permit Counter Report to find parcel information, Services access application/permit notes and inspection activity or Department environmental compliance monitoring, and access easement documents and development reports 39 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 275 of 461 Page 42 of 195 • The Oracle Public Sector Community Development module requires GIS Map services of address points with sewer mains and parcel information • Mainline review sends Developer Jobs for GIS/Asset Management review to ensure data spatially aligns with current assets • Provide sewer renovation risk rankings, recommended sewer renovation rates, and budgetary estimates using InfoAsset Planner • Provide flow data and perform capacity evaluations using InfoWorks and a small diameter hydraulic capacity tool • Develop and carryout new programs and inspection projects aimed at generating additional likelihood of failure data for assets and improved logic for understanding asset risks • Recycled Water Planning group tracks inspections using Cityworks • Asset Management and Maintenance staff engages with Capital Projects Division staff to update the asset registry for the acquisition and disposing of capital assets for both Collection System and Treatment Plant capital projects • Survey in Capital Projects Division provides scanning data, orthophotos, and topographic features and linework • The Environmental Compliance GIS database of businesses in Central San's service area Is hosted and managed by Asset Management • Engage with Regulatory Compliance Group to continually update assets that are tied to various compliance regulations such as Title V, Title 22, NPDES, and Bay Area Air Quality Management District (BAAQMD) • Regulatory Compliance Group uses the Title V Instrument Work Order Report to ensure all instruments used to demonstrate compliance with Title V requirements are being routinely calibrated • Engages with Laboratory work group to update laboratory assets in the asset registry Operations • The CSO, Plant Operations, and Maintenance Divisions use CMMS for planning, scheduling, and executing work and report analysis • CSO CMMS to schedule cleaning, record cleaning results and CCTV data, map updates and create work orders for scheduled and unscheduled maintenance • Plant Maintenance uses CMMS for planning and scheduling, Failure Reporting Analysis and Corrective Action System (FRACAS), Predictive Maintenance (PdM) and Asset Condition Management Programs, monitoring, and reporting • Treatment Plant Operations uses CMMS to create work orders and inspections. In progress, Asset Management is engaging with Operations staff to digitize Operator Rounds using new CMMS interface • Plant Maintenance has a project to migrate Pumping Stations to use CMMS • Continuous training and development for staff All staff use GeoPortal, which consists of web applications for the entire Central San to use and search for asset management documents. 40 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 276 of 461 Page 43 of 195 C. EXTERNAL STAKEHOLDERS Stakeholders Asset Management Engagement Rate Payers Provide maps such as Pharmaceutical Drop-off locations and service area maps Water and Wastewater Organizations Participate in organizations related to asset management or GIS discussions such as Esri's or Cityworks regional user group, Esri's Leadership Utility Enterprise Agreement User Community • Participate in Bay Area Clean Water Agencies (BACWA) InfoShare group and Asset Management Professionals Partner Agencies Develop working relationships to exchange GIS data, ideas, workflows, and strategies Consultants Provide data and maps as requested and work with consultants on projects D. STAKEHOLDER COMMUNICATION PLAN COMMUNICATION GOALS AND OBJECTIVES Communication across the organization and externally brings awareness to assets and asset management systems. • Have a common understanding of the purpose of asset management and its processes and acquire the necessary resources for the success of the program • Understand internal and external stakeholder expectations • Communicate Asset Management progress, risks, and related information • Promote Asset Management awareness • Facilitate/promote cross-functional collaboration • Inform and educate process for gathering accurate data/information • Communicate and work together to ensure alignment and consistency with Central San Strategic Goals and Asset Management Policy • Avoid duplication of efforts and data and avoid departments working in silos • Avoid creating multiple asset registries • Communicate when performing system and software upgrades or improvements and provide ample time to alert of potential outages . . Description Email cityworks@centralsan.org Email for Cityworks and GeoPortal geoportal@centralsan.org support, data updates, and any requests 41 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 277 of 461 Page 44 of 195 Meetings User Group or stakeholder Monthly meetings with meetings to discuss Asset stakeholders and Asset Management Management workgroup Plant Maintenance meetings with the Warehouse Training (group Invest in People and Training Ensure staff has an informed and or individual) and provide video training to confident workforce about and Training Central San's video GeoPortal and Cityworks Video distribution workflows Cityworks Tools Dropdown menu in Cityworks User-requested reports for assets, & Reports linking to applications such reserved assets, KPIs, benchmarks, as Spare Parts and reports to map updates and others download Dashboards Dashboards with indicators, Distributed to group members in KPIs, and benchmarks for divisions CSO and Treatment Plant San Central Internal Central San website Website to distribute general website information and any frequently asked questions 42 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 278 of 461 Page 45 of 195 ASSET DESCRIPTIONS 1 . EXISTING FACILITIES A. GRAVITY SYSTEM The gravity system has about 1,500 miles of pipe, ranging in diameter from 4" to 102". Central San owns 76 miles of wastewater trunks and interceptors ranging from 24" to 102" in diameter, which accounts for 5% of the collection system. The typical lifespan of large-diameter sewers ranges from 50 to 150 years, depending on pipe material, hydraulic, operating, and environmental conditions. Most sewers are 8" in diameter, with an aggregate length of 68% of the total system. Clay and plastic materials are the most common, each accounting for about one-third of the system length. Concrete pipes make up one-quarter of the system length, with ferrous metal pipes at less than 10%. The oldest sewers were constructed in the early 1900s and are still in service. However, overall, the system is relatively young, with more than 50%installed between 1960 and 1990. B. FORCE MAIN SYSTEM Central San maintains 31 force mains with a combined length of approximately 24.8 miles ranging from 1" to 30" in diameter.The smaller force mains, 1" to 4" in diameter, are located in northwest Orinda, where several parcels have private pumping/lift stations. More than 65% of the force mains consist of metallic materials prone to corrosion. The typical lifespan of force mains ranges from 50 to 100 years, with over half of the existing force mains installed 40 or more years ago. 15� PWIR "' I71 Figure 11:CSO Crew Cleaning a Sewer Main with a Vactor Truck 43 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 279 of 461 Page 46 of 195 C. PUMPING STATIONS Central San owns 15 pumping/lift stations and maintains an additional three by contract. The pumping stations range in-service requirements from only one residence and a few hundred gallons per day to 10 MGD in dry weather. r f . Figure 12:A Central San Pumping Station D. TREATMENT PLANT The Treatment Plant includes a Headworks influent pumping, pre-aerated grit removal, primary sedimentation, primary effluent pumping, activated sludge secondary treatment, and UV disinfection. The treated effluent from the treatment plant is discharged into Suisun Bay or treated further for recycled water use. Tertiary filtration produces Title 22 recycled water for on-site use, irrigation, and other recycled water uses within Central San's service area. The wastewater treatment plant's solids processes consist of secondary sludge thickening with dissolved air flotation thickeners, centrifuge dewatering of blended primary sludge and thickened waste-activated sludge, and incineration of the blended sludge. Incinerating blended sludge requires cofiring with natural gas or landfill gas. Approximately 90% of the treatment plant power demand is satisfied using onsite cogeneration with a 3.5-megawatt natural gas turbine, and the remaining power demand uses imported grid power (electricity). Waste heat is recovered from the incinerator and cogeneration turbine. The waste heat generates more than 70% of the treatment plant's steam demand. Auxiliary boilers supply the remainder fired on natural or landfill gas. Approximately 90% of the plant's steam demand drives the aeration turbines that provide oxygen for the secondary treatment process. 44 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 280 of 461 Page 47 of 195 E. RECYCLED WATER The treatment facilities include coagulant addition, sand-anthracite filters, chlorine disinfection, and storage. The distribution system was constructed in the late 1990s to serve nearby Concord and Pleasant Hill areas. Additional service connections have been added over time, and the mains were extended in 2012 to serve additional customers in the Concord area. Central San owns and operates about 13.8 miles of active distribution piping ranging from 1 " to 24" in diameter. Thirteen miles of distribution piping ranges from 2" to 42" in diameter. The majority is concrete mortar lined and coated steel or polyvinyl chloride (PVC). F. GENERAL IMPROVEMENTS The General Improvements Program primarily concerns property, administrative buildings, and management information systems. These include office buildings, fleet, IT infrastructure, and software. Buildings or equipment dedicated to the collection system operations or treatment plant are included in those CIB programs for funding purposes. 2. STATE OF THE ASSETS A. COLLECTION SYSTEM An evaluation of the condition and performance of collection system assets was completed in 2016 with the CWMP (Carollo Engineers, 2017). Additional details can be found in: • TM CS-14, Collection System Condition, and Risk Assessment • TM CS-20, Capacity Analysis Results Overall, the gravity system is in good condition with a low Likelihood of Failure (LOF), as shown in Table 14.7 of TM CS-14 and republished below: . • • • 1Very ®�• : . • • ■ 1 - Like New ■3-Good . . . ■4-OK ■6- Fair ■8- Poor I Like New, ' ■9- Bad ■ 10-Very Bad Figure 13:Condition of the Gravity Portion of the Collection System 45 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 281 of 461 Page 48 of 195 Central San completed additional pumping stations condition assessments from November 2015 through early 2016 as part of the CWMP (Carollo Engineers, 2017); for more detailed information about the program, see TM-CS-14, Condition and Risk Assessment, Appendix A. These comprehensive condition assessments identified several structural, mechanical, electrical and instrumentation improvements. Thus, to address aging infrastructure and reliability needs, Pumping Station Upgrades, Phase 1, replaced the electrical, mechanical, and backup power equipment at the Moraga, Flush Kleen, and Orinda Crossroads Pumping Stations. The project upgraded electrical and controls for the new electrical wet weather pumps, including new switchgear and variable speed drives; upgraded backup generators and automatic transfer switch improvements; replaced wet weather diesel engine driven pumps with electric motors; reconditioned or replaced pumps, valves, and gates; replaced/added surge tanks; replaced six dry-pit pumps with new submersible pumps, and added grinders to the Moraga Pumping Station. Pumping Station Upgrades- Phase 2A will begin for the needed structural, electrical, and mechanical equipment replacements at the Maltby and Fairview Pumping Stations. The scope consists of the installation of two new 12.4 MGD sewage pumping stations and includes, but not limited to, heavy civil, mechanical, electrical, instrumentation and controls, installation of new PG&E services, and demotion and decommissioning of the existing pumping stations. Phase 213 will be electrical and mechanical replacements at the Martinez Pumping Station. San Ramon Electrical Upgrades will address electrical deficiencies identified at the San Ramon Pumping Station for the daily operation of powering wastewater pumps or to power the entire pumping station during PG&E power shut-off conditions. The electrical equipment includes several variable frequency drives, the automatic transfer switch for standby emergency generator power, and the main electric service entrance for the pumping station. In February, an annual report on sanitary sewer spill is presented to the Board. There have been no significant operational issues over the last three years, and sanitary sewer spills were reduced over the previous ten years. In the calendar year 2022, Central San had 23 spills, which averaged 1.49 spills per 100 miles of pipeline. Central San's sanitary sewer spills rate is below the statewide average of 3.4 and the Bay Area peers' rate of 4.7. Central San has held an average of 31 spills per year over the last 8 years. Situations arise where Central San cannot predict the spills. For example, heavy rainfall during the winter of 2016-17 caused a storm drain failure in Orinda that also washed out a road and underlying sewer, which is not a situation that Central San can prevent. These situations occur periodically, and Central San has emergency response procedures (see Operational Resilience and Disaster Preparedness) to address them as quickly as possible. CSO also uses the "Sanitary Sewer and Backup Response Plan" as required by the Waste Discharge Requirement (WDR). 46 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 282 of 461 Page 4• of • ••le 5:Total Spillsby - TOTAL SPILLS BY YEAR (ALL CATEGORIES) 47 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 283 of 461 180 160 143 162 140 14.1 1 120 123 00 10 100 81 80 73 63 60 49�4 43 U U 49 4� 40 40 —O/ i O 38 28 22 24 26 20 37 U U U 0 0 23 0 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016 2018 2020 2022 2024 Page 50 of 195 B. TREATMENT PLANT The treatment plant has the highest level of risk due to the poorer aggregate asset condition.This is expected since equipment assets have service life estimates of 10-40 years, whereas piping assets in the collection system range from 75-100+ years. Furthermore, most of the treatment plant was constructed in the late 1970s, so it is natural timing for many mechanical assets to require replacement. The most recent condition assessment was completed for the Steam and Aeration Blower Systems Renovation Project (Steam Project). Another summary of existing facilities was completed in 2016 with the CWMP (Carollo Engineers, 2017) in TM TP-5, Existing Facilities, and TM TP-9, Condition Assessment. The capacity evaluation in the CWMP reviewed capital needs over 20 years. That study found the need for facility improvements will include both capacity and infrastructure upgrades to account for additional wastewater and wet weather flows. Wet weather improvements are needed regardless of population growth to provide reliable collection and treatment of wastewater through severe storms. The treatment plant was primarily built in the late 1970s and modernizing it will include investing in new technology and addressing increased energy and power demands. To ensure long-term reliability and efficiency, the assessment concluded: • Not necessary to build a new treatment plant. The plant has a solid foundation and will require primarily mechanical, electrical, and instrumentation improvements to address aging assets that are in poor condition, unreliable, or increasingly costly to maintain. • New solids/energy projects will be developed to ensure Central San provides reliable solids handling facilities and invests in sustainable solutions while reducing its energy footprint and general operation and maintenance costs. • UV System Replacement with modern controls to modulate electrical demand on flow and water quality characteristics, and built-in cleaning wiper blades to reduce staff time while maintaining efficient performance. • Improved instrumentation and power monitoring are recommended to help identify opportunities to reduce energy demands and other operating and maintenance costs. Capacity-related projects for the treatment plant include: • Filter Plant Expansion: Add a chlorine contact tank and expand support equipment. Add filter pretreatment upstream of the filters, optimize existing filters, or install new tertiary membrane filtration to meet Concord Community Reuse Project Title 22 recycled water demands when they occur. • Secondary Treatment Hydraulics: Add secondary clarifiers to increase peak hourly wet weather flow capacity and a mixed liquor splitter structure to split between clarifiers evenly. • UV Hydraulics: Add low lift pumps and a parallel final effluent pipe to the UV channel. • Primary Expansion: Add two primary sedimentation tanks and a pre-aeration tank to reduce surface overflow rates and improve performance. 48 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 284 of 461 Page 51 of 195 Current Treatment Plant Projects to address the above assessments: • Most of the current UV system has been in service for 26 years and requires substantial maintenance, lamp replacement, and frequent chemical cleaning of UV banks. For wet weather operations, the UV system only has a capacity of approximately 90 to 100 MGD instead of 127 MGD. The reduced capacity creates a hydraulic bottleneck in the secondary treatment processes, limiting the discharge of disinfected final effluent during peak wet weather events. District Project (DP) 100012 (UV Project) will execute an agreement for the planning and pre-design services for the UV Project. • Aeration Basin Diffuser Replacement Project (DP 100019) will increase process effectiveness and ideally reduce sludge volume index, leading to more capacity in existing clarifiers. Bringing south selector back online will allow for step feed operation maintenance on north selector which has been in service since the 1970s. North mixed liquor channel gate replacement will provide operations with control of flow split to north and south clarifiers. Central San has met 100%compliance with our NPDES permit for 25 consecutive years and was recognized with a Platinum Peak Performance Award from NACWA. C. RECYCLED WATER The Recycled Water program can be divided into two main categories: treatment and distribution. The most recent summary of existing facilities was completed in 2016 with the CWMP (Carollo Engineers, 2017) in TM TP-10, Recycled Water. The distribution system has pipelines ranging in age from 3-42 years. Assuming a useful life of about 60 years, the system is in relatively good condition. However, the Recycled Water Model Update and Hydraulic Analysis (West Yost, 2019) noted that the recycled water distribution system was experiencing transient pressure in some areas of large demands due to system configuration. Staff have begun to assess isolation valves in the distribution system and believe that there are some valves that are in poor condition that will require replacement. Most tertiary treatment facility assets are in fair to poor condition due to their age and shorter service life. The distribution system is in good condition, with the service life being longer on average, and installed more recently than the treatment facilities. The CWMP recommended improvements to the filter plant and clearwell due to aging assets, and recommended expanding the filter plant to meet near-term demands. On May 3, 2018, the Board authorized an agreement with Stantec Consulting Services, Inc. (Stantec) to provide design services for the Filter Plant and Clearwell Improvements Project. Stantec reviewed the facilities' condition and provided disinfection alternatives, hydraulics, seismic assessment, process and automation solutions, and alternative filtration technology information. Staff and Stantec took a long-term approach to this project and evaluated the possible impacts of future projects, including the Concord Community Reuse Project and the Water Exchange Project. The team assessed Central San's immediate needs when looking at possible solutions. The needs included the most critical items that require replacement to restore reliability but would not conflict with or potentially leave stranded assets when other projects are implemented, like the Water Exchange Project. Several recent failures have shaped and focused on staff recommendations for recycled water facilities. The east clearwell cover experienced failures leading to a repair project that included decommissioning the east cell and making improvements to allow the west cell to operate. In 2018, a high-voltage (12 kilovolts (kV) electrical 49 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 285 of 461 Page 52 of 195 feeder from Substation 82 (Central San's main PG&E interconnection) to Substation 40 at the Filter Plant failed. Substation 40 experienced another failure in early 2019, which triggered a cascading series of events that led to a 16-minute, plant-wide power outage. These incidents reflect that, despite the staff's good operation and maintenance practices, the recycled water facilities are showing signs of significant deterioration, which is starting to impact reliability. Staff recommended dividing the Filter Plant and Clearwell Improvements Project into three phases, with the most critical elements proceeding first, as follows: • Phase 1 A (immediate replacement needs): Electrical renovation, recycled water storage tanks, clearwell pump improvements, rehabilitation of one of the four existing filters (additive bid item), and Basin C modifications. • Phase 1 B (within 5-10 years): Rehabilitation of the remaining three filters and the Filter Building, other electrical replacement, upgrades to the Applied Water Pumping Station, hydraulic improvements to bypass the Forebays, and adding a chlorine contact basin. This phase would complete the remainder of the renovations to meet the long-term needs of the plant and Zone 1 customers. • Phase 2: Improvements to provide additional recycled water supply to serve the future Concord Community Reuse Project. Repair and replacement for the recycled water distribution system are being evaluated. There are no major projects currently underway. D. GENERAL IMPROVEMENTS The General Improvements Program is primarily concerned with property, administrative buildings (Headquarters Office Building, Annex, HHW, and Bays), management information systems including IT, asset management, and new equipment and vehicle needs of Central San. Improvements were made to the Headquarters Office Building in 2013, and a new CSOD building was constructed by the summer of 2012. Seismic improvements were also made to the Annex and HHW buildings. Most of these assets are in good condition. Replacements or improvements are scheduled in the CIP. E. CAPITAL IMPROVEMENT PLAN PRIORITIZATION - LINEAR ASSETS Gravity sewers were analyzed using InfoAsset Planner analysis software and completed in 2020 for the CWMP (Comprehensive Wastewater Master Plan) (Carollo Engineers, 2017) and described in more detail in TM CS-14, Collection System Condition, and Risk Assessment. This InfoAsset Planner model combines GIS data for the gravity sewer pipelines with CCTV results and maintenance records to calculate the LOF scores for each pipeline. Additionally, the model assigns COF scores for each pipeline based on other risk factors such as proximity to railroads and surface water. The scoring approach used in the model was developed in a series of workshops conducted between February 2016 and July 2016 and has been subsequently enhanced to reflect improvements and feedback from staff. 50 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 286 of 461 Page 53 of 195 In FY 2017-18, the Planning staff implemented the system on in-house servers with the assistance of GIS staff. The Associate Engineer in Planning is the system steward, with support from the GIS staff, to establish the data links to the GIS, CMMS, CCTV, and Hydraulic Model. Figure 14 shows a screenshot of the gravity system prioritized for replacement. 0 Sewer Engineering k, i . 0 ; Renovation Model Timeline > 00-05 Years „� .`. W:Itl 1P#Creak 05-10 Years :, 1 , 10-20 Years {"1 t 20-30 Years _ `.. f,..oc��, °' • - 30-50 Years 50-75 Years 75-100 Years _ - _ �'•i ` .� 100+Years � �" ^' I - -�• 0 ; Large Pipe Corrosion — .� l i, r - d + ❑I Sewer operations i * 0 Sewer a . = _ f -f- 0 Recycled Water ' '• + ❑ Parcels " 1 Saran. ►►Layer=- Default ,vim• 0 0. � . ® Y .- .2nFi Central San,Contra Costa Coun Figure 14:Capital Improvement Plan Prioritization Screenshot F. CAPITAL IMPROVEMENT PLAN PRIORITIZATION - VERTICAL ASSETS The CWMP (Carollo Engineers, 2017) identifies a summary of the immediate and future needs for improvements to the wastewater treatment plant in TM TP-12, Immediate and Future Needs. The requirements were derived from the following: • an evaluation of the baseline treatment performance, • the physical condition of major facilities, • projected flows and loads, • the treatment capacity required to meet current discharge requirements, and • the capacity to meet future discharge regulations for effluent total inorganic and nitrogen levels. While TM TP-12 has provided direction to the Engineering &Technical Services Department for future capital investments, a prioritization model was not developed that could be handed over to Central San staff. Since needs and constraints may change over time, the Asset Management Section should develop a simplified strategic forecasting tool using existing software tools and data from the asset registry and the CMMS. A conceptual process map for this tool is shown in Figure 15. 51 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 287 of 461 Page 54 of 195 Asset UID ALGiin6 ' Install date Irrfrostructure - - ' - Overhaul gate Service Life{yls} Cond rtion Scare(1-100) Cond rtion date OOF(1-10) Yrs a I in Re6rlatorg LOF(1-10) tp RRE(1-100) Add:Repl Driver(Aging Inf,Sustain,Reg,Capacity) N3 Ada:End of UF&(EOL] Ada:Risk Category Add:CIP Bucket LOF= LOF=[arerhaul LOF-Paodwr OF=manual EOL= OcNwttdriwmr is a*g irrframucture. CarndWon dot=+fif,Servi¢ imstallko krwi¢ J10 Life'365))'10 Life-365)'is entry hbnuvl Eirdy For FlectTahks.use Purchase datr inroad of Inch Date_ ff EOL K today,thnr LOF==•s EI]L= EOL=tndap+ LOF=iIJEOL - Condition Oaf EOL�rhaul EOL=iinstall LflFf -T� ] -LOFJ10 dwu!- ,Serviur d9t&I-kru�e IL•�GFf I,-TDda])/ Senior L FL-3651 Lirir'365 Lirr96rk '3 vicz Li Fe LiFe966 uWdraad of Pallun[l04F Lo, NL law Me&" ril lr Ff�S H�Ir Mrtlirt R isk C=epxW Za (see Fig 14111 W.►licr wdiws ulydri.r A rd6yi- to" IJr 11 mrk.. 4 M.Low low low rArEwm h'tEi.rn u° l4A N.ylipbIV Lbw LRw WdI n Fqur ul.11 t a a Figure 15:Conceptual Process Map for CIP Prioritization for Vertical Assets 52 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 288 of 461 Page 55 of 195 ASSET MANAGEMENT INFORMATION SYSTEMS AND TOOLS 1 . DATA AND KNOWLEDGE MANAGEMENT Central San implemented an enterprise GIS-centric and reliability focus Asset Management system. Staff aims to systematically capture assets and their attributes in an enterprise relational Structured Query Language (SQL) database and integrate business systems whenever possible. This section describes software platforms, data infrastructure, how staff models data, and data management strategies. 1AIbr1 IlOY� y InfoMaUer" DYNAC Plant COnlr Ul I lydraulk Ca adty ask Model Asset Malnbenanr•e P System{SCADA) Model Prloritlred Capital Plan Manegemen[ I I �I. Arc GIs Y EMarprlsa ® Oa[a AutomeUCli CRACL Gcudatahases and Map Semmes Enterprix leriorrr►Punnr.IK j 11 �CIC�� CUdlO ••••...•••• tM ! ,ware ,k�ayy 3�Lkx Web Mapping Appik nt {1211iffi4mdal —10 16 ++L Errmk e+�nf.ew+r.b "' r4n�.m.,a iaal I } Spare Puts µ OIG-SMART Mamigement Tool Da men[ USA N—h Ell L—al s� .w• � Mana�rment � Data flaw Web hyperlinks �..... Map Services Figure 16:GIS Platforms,Infrastructure,and Data A. GEOPORTAL GeoPortal is an invaluable resource for staff across the organization. GeoPortal has two main sites: (1) Central Portal focused on the horizontal assets (collection system and recycled water distribution), and (2) Treatment Plant Portal focused on the vertical assets (treatment plant, pumping stations, and general facilities). Staff can click on a map feature to display its details, as shown in Figure 17, and access external databases such as record drawings, inspections, and maintenance history via embedded hyperlinks. In addition to panning the map to view assets or selecting the features to access linked datasets, staff can also use the search tool by entering a whole or partial address, parcel number, or facility ID. Each asset has a GIS ID automatically assigned by the system to ensure all assets have a unique identifier. Each asset also has a facility ID, a semi-intelligent naming system that concatenates the Central San's map grid, structure type, and structure number to create a common name for the asset familiar to staff. For example, a facility ID of 75A2-M47 indicates a maintenance hole 47 is on Central San map quadrant 75A2. 53 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 289 of 461 Page 56 of 195 Maintenance crews can view pipes to target cleaning needs, provide quality cleaning, avoid issues such as stuck rodding rods, identify potential hazards, and evaluate causes and options for dealing with spills. Engineering staff uses the information to prioritize and design capital projects. Permit Counter staff access current information to assist the customers with permits and respond to inquiries. Central Portal 75a2-n47 Central San s Access to Geospatial Information Navigation Tools Links o r a 13 ul OP. s Contact and Initial View Previous Extent Zcci n In Zoom Out Pan Bookmarks Plot Coordinates Open Map Disclaimer Legend Legend = x < ✓ also Filter Swatches".. ©�il[er l� �`sewer Main X • Flushing Inlet-Actve `+ Facility ID:75A2-M47/75A2-M27 Fee:51— n 7 Material:Vitrified day • Lamp Hale-Active — Diameter.6' ;eee s Length:291 X Structure Abandonment Scheduled Status:Active M49 Jabs:%1552 Nq M2a a • Structure Unlocated — e S� View Additional Details Addto Results • MULPSS Flushing Structure-Active o M49 Fuel Oil q. g Storage. O MULPSS Flushing Structure-Tentative ' — 'P y Emergency Tank _ w Generator Main III M47.4 W a- `c Lon nM71 — Sewer Main - •'"" �~� -- CPS 15' M47 Acdve 6 Collection System Renovati on Scheduled c3 <..f''� �} ceparcmem euilaing r "." Tentative Abandonment Scheduled l -" Inactive or Future Use M70 — Sewer Casing - q Collemon ystem Q Mti6 1 O ra'ons Pl pe y,g Legend Default yr 0®OH 1500�] Yr rA Sm ini,ra. G + Central San,Contra Costa County Central San I Bureau of Land Management, Figure 17:GeoPortal Web Application Additional reports linking to map features include: • Asset Details • Record Drawings • Right-of-Way/Easement Documents • C&M Manuals and Shop Drawings • Environmental Compliance Inspections • Permits • Condition Inspections/Assessment Results (i.e., CCTV Data) • Assessor Parcel Maps • Maintenance Records • Hydraulic Analysis Results GeoPortal allows staff to turn on/off combinations of layers and base maps to display their needed information. Map layers available through GeoPortal include: • Collection system assets-including structures and pipelines, and pumping stations • Treatment Plant civil, mechanical, electrical, instrumentation, and controls assets • Waterways 54 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 290 of 461 Page 57 of 195 • Roadways • Administrative boundaries • Parcel boundaries • Right-of-Way and easement boundaries (as available) • Other agencies' utilities (as available, such as water, stormwater, and power) Datasets or reports associated with map features include: • Parcel-related information includes: o Permit records o Rates and Fees o Assessor parcel information o Environmental Compliance activities such as source control program inspections o Right-of-Way and easement documents Base maps available through GeoPortal include: • Aerial imagery • Topographic maps • Street view Central Portal Cenaal 5-n A—m C—patial Inf—int n Navigation Tools Links + 0 C.—and Initial View P—i-12— Zoom In room0. Fan 13-ii-A. Pbicoordina[es Open Nap Di.lai Legend G° c Reev F.ac Sewer Main X { S I a� •q3 - 01 Mz! a Hyperiinks �Iaa� s Sys lob ReportlRecord U—nin0 T M49 g� OP Fto S Sx J!J !me*gYlV rank CCN Ins e _ _ ❑ / r�b� C V Grr /// M47.4 1Y"� rsMA MaintenanrP Informayon ' p� 6 Sit 5 Asset Management Documents (,F s2j M47 6 Calletnon SYrrem Operations ICSOI Details oenanment suo", Facility Identifier 75A2-M47p5A2-M27 r A se Id 69392 4 KA, Active 4 t Status Actve �or.eena syu•�n oce�ra'�"'�ra�nsF0.ioe r 11 oats •� a' Sho[to re 01101/1W N ❑iamexer 6. X M25 Material Vitrified Clay !J 7 Recorded Le 7th M59 P 291 `sI Job Numbers K7552 M51 s M74 8+yl:p- p-ia 711 � pR35 %p V ihO i;IAISLO pumping Station'con= II WSJuy(.Y977R 3rllP � C e.YLP-a6R 7 r 1dlNp prAgLn M25 — r to '�.Leg— R- Sewer Mai- Oe7au l: g®t L—ml San,Contra Costa CounTy Cen4a1 Figure 18:GIS Additional Details for Asset Another tool within GeoPortal allows staff to plot the coordinates, i.e., the mouse position's latitude and longitude values, which staff can use when reporting sanitary sewer spills to the State Water Resources Control Board. 55 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 291 of 461 Page 58 of 195 B. MAPPING CUSTOMIZATION FOR FIELD STAFF During the replacement of the GIS software, GIS staff conferred with CSO staff and other workgroups to develop map "themes". These "themes" tailor the viewable layers and labels to meet the needs of the individual work groups. In addition, feedback was solicited on symbology and labels so the most valuable attributes are easy to view by field staff. For example, the label along a pipe displays the diameter, material, and length of the pipe segment from structure to structure, is valuable information for cleaning crews. C. MAP UPDATES GIS staff updates the collection system maps with record drawings from Capital Projects and Central San inspectors. Construction inspectors submit the mainline extension projects. Changes are usually made within a few months of completion of the work, but the maximum update interval goal is one year. Other datasets, such as aerial imagery, are updated as they become available. Any staff can submit map updates via a typical e-mail using the address geoportal@centralsan.org or from web forms accessible within GeoPortal that the GIS group monitors. Field staff can also submit map update requests through the CMMS. Work orders include a field where staff must select yes or no for "GIS Update." An inbox for these submissions is monitored by GIS staff, who then update the GIS accordingly. The CMMS software maintains an audit log containing the date, time, and staff names. D. ASSET INVENTORY: SINGLE ASSET REGISTER As stated above, in the late 1990s, two full-fledged CMMS programs were for different workgroups. The Plant Operations Department (POD), responsible for operating and maintaining the facility that cleans an average of 34 million gallons of wastewater per day, used Mainsaver. The CSOD, responsible for cleaning, maintaining, and repairing the over 1,500 miles of underground pipeline that makes up Central San's sewer system, used a Sussex program that Accela later purchased. This version of the software did not have mapping capabilities, so the GIS and mapping team maintained a separate dataset of the collection system from which to create maps. Keeping the two datasets synchronized and reconciled proved difficult, which led to inconsistency in the data. These data inconsistencies, the customizations in the CMMS program that forced version lock, limited functionality, data duplication, and historical work orders/inspections connected to incorrect assets were persistent problems. Moreover, the CMMS that CSOD used was no longer vendor supported. Central San chose to consolidate work management and migrate into a single GIS-centric, vendor-supported solution to manage both systems and staff resources effectively and integrate different software platforms. The solution allowed integration with additional programs and tools to expand the AMP beyond essential maintenance management and data capture. Today, Central San has a single asset registry for Collection Systems, Treatment Plant, Fleet, and Recycle Water Distribution. The asset registry is the foundation for all asset management practices and is a live tally of all Central San's physical assets. The asset hierarchy shows relationships between assets in the registry, whether by process or physical relationship. Agencies may have multiple asset registries amongst different departments, developed over time, requiring custom integrations and continual reconciliation efforts to keep them updated and synchronized. As a 56 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 292 of 461 Page 59 of 195 result, metadata issues arise with multiple terms used for the same assets or various names for different work categories. An important goal was to improve business processes and reduce data errors by reducing registries and CMMSs to a single database. Central San maintains a comprehensive asset registry to facilitate asset lifecycle management. Attributes were added to the asset registry to track the asset cost, condition, LOF, COF, risk, and other data used for asset management. Central San continually identifies, collects, and provides comprehensive Quality Assurance/Quality Control (QA/QC) screening of relevant data and applies sufficient analyses to turn that data into knowledge that supports sound, evidence-based decisions. This asset registry is in a Structured Query Language (SQL) relational database with versioning, multi-user editing, and different editing privileges. The Collection System data model uses Esri Local Government Information Model (LGIM) based on the Geometric Network for horizontal assets. The LGIM incorporates many best practices for water and wastewater utilities and is configured to Central San's needs. The LGIM makes deploying and updating maps and applications more efficient and facilitates sharing data with partner agencies through standardization. The geodatabase design uses object tables and relationship classes for vertical assets like pumps, motors, and valves at the treatment plant. E. ASSET HIERARCHIES A hierarchy organizes assets to help budget, operate, and maintain those assets. The Capital Improvement Budget (CIB) consists of four programs: Collection System, Treatment, Plant, Recycle Water, and General Improvements. The hierarchy is oriented to allocate assets into those four programs by process, sub-process, system, and asset class, as shown in Figure 19 in the schematic below. Process Hierachy Geographic Relationship Entity Group: Treatment Parent Feature Plant Process Parent Asset Sub-Process Child Asset System Component or Spare Part Figure 19:Schematic of Process Hierarchy and Geographic Relationship The Entity Group is set by the geodatabase schema developed by the GIS staff. Assets shall be in one of the following Entity Groups: • Collection System • Recycled Water (ReW) • Fleet • Treatment Plant (includes ReW treatment facilities, pumping stations, and facilities) 57 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 293 of 461 Page 60 of 195 F. PROCESS HIERARCHY The TPFleet database is the asset database for the treatment plant, pumping stations, and fleet assets. For this database, attributes are added to each asset to aid in searching, querying, or analyzing assets based on a process hierarchy. For this, an overall hierarchy was created and applied to each asset, shown in Figure 19 and Figure 22. Figure 20 shows a report developed to list the assets in each pumping station, filtered on the Process and Subprocess attributes. A similar report was developed for the laboratory with specific attributes they were interested in seeing, as shown in Figure 21. CENTRAL SAN Pumping Station Asset Registry for San CENTPAL CONTRA CO&TA SANITARY DISTRICT Ramon Pumping Station (SRPS) Asset No Name Location Asset Class Lifecycle Status Breaker 1600A SRPS Generator-MCC Rm Circuit Breaker __ In Service Control Sys HMI-San Ramon PS SRPS VFD Rm CPA tems In Service Breaker f1600A) SRPS Generator-141CC Rm Circuit Breaker In Service _Br dge Mux Gateway-SR PS SRPS VFD Rm Control Systems __ In Service Breaker(1600A) SRPS Generator-MCC Rm Circuit Breaker In Service 120000 San Ramon Pump Station 9925 Mangos Drr San Ramon-,Crass Ens Site In Service 120071 HVAC,Control Rm SRPS Office HVAC Equipment In Service 120073 Refrigerator SRPS Office _ Appliance In Service 120089 Intrusion Control Unit SRPS Office Security Equipment In Service rity 120090 Secu System�R SRPS Office Security Equipment In Service 120098 UPS,SCADA SR SRPS Control Pnl SR-CPA UPS In Service 120099 Workstation-San Ramon PS SRPS Control Pnl SR-CP-2 Control Systems In Service Figure 20:Sample of an Asset Report for a Pumping Station col CENTRAL SAN Laboratory Asset Registry AsselNe, Name Install Bate Location Asset Class Condition Condition Date Process SubProcess System Asset Cost 22231 Sampler,Pn Eft(Lab) 1 211 612 01 4 Pri Sed Basins West-End(Lab) Sampler 10 12P612014 Support Systems Equipment Lab $2,000.M 26050 Ioftent Sampler 06114120M Headworks East Entrance Sampler 100 818/2014 Support Systems Equipment Lab $2,000W 31400 FE Sampler hl o ler Pump 032V2013 AM Tanks S End Sampler Too 818/2014 Support Systems Equipment Lab $1,200.M 31406 Sampler,Secondary MV2013 AM Tanks W Side Sampler Too M014 Support Systems Equipment Lab $1,000.M 31900 FIE Sampler 17J022014 AfN Tanks S End Sampler 100 8182014 Support Systems Equipment Lab $1,482 00 321390 Pum p,HR Booster(1i C3199f2004 T7 Pump 100 818/2014 Support Systems Equipment Lab $6,600.IX) 34510 Bioassay Trailer(Lab) 0510112005 S Quad(Lab) Building Feature 70 8182014 Buildings Lab Bioassay $102,000 00 34E12 Sample Pump,Bioassay Trailer 051012005 Bioassay Trailer Sampler 60 8182014 Support Systems Equipment Lab $3,000 00 34520 HVAC,Bioassay Trailer 0510V2005 Bioassay Trailer(Lab),S Quad HVAC Equipment 40 M014 Buildings Lab WAC $5,000.M 34522 XFMR(Dry)-Bioassay 01101119% Bioassay Trailer Transformer 60 8/3/2014 Buildings Lab Electrical $7,500 W 34525 Tank,Test System,Bioassay 13719V1977 Bioassay Trailer(Lab) Shop Equipment 70 8182014 Support Systems Equipment Lab 5500.00 36000 Laboratory CWDB2001 Secondary Area(Lab) Building Feature 20 818/2014 Buildings Lab Strui 55,065,000.IX7 35001 Exh Fan,East,LEF 1 0912212001 Lab Roof Fan 30 8182014 Support Systems Equipment Lab $2,000.M 36002 F Fan,East,LEF 2 091222001 1 Roof Fan 30 8182014 Support Systems Equipment Lab $2,000.11 Figure 21:Sample Report for Assets in the Lab 58 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 294 of 461 Page 61 of 195 Recycled Collection Primary Secondary Disinfection & Filter Plant& Solids Support Pumping Syst Water Treatment Treatment Final Effluent ReW Handling Utilities Systems Stations Buildings Sitework General Distribution Headworks Primary Effluent UV FE Diversion Sludge Thickening Power Generatior' Equipmen} Acacia 4737"Bays" CSO Campus •Influent Pumping Primary Effluent Channel UV Basins Applied Water DAF Polymer Cogen B&G(Buildings& Building Electrical Landscaped Area District Properties Force Mains Influent SZ ning Primary Effluent UV Cleaning Forebay DAFTs Stantlby Power Ground)Shop •Equipmen} •Elevator Non-Landscaped Area •Influent Structures Pumping Influent Valve Structure Electrical Shop Piping Fire/Life/Safety/Security Roads&Parking •Engineering Sitework HVAC Security/Fencing/Gates •Household Hazartlous Interior Finishes Storm Drains waste Flee} Sludge Blending Fuel System •Plumbing •sneer fighting Hypochlorife Instrument Shop •Roofing •Equipment Headworks Buildin Aeration/Nitrification Filters Carbide Lime System Fuel Oil Laboratory •Mobile Equipment g Hypocnlorl Site I .ESST •Landfill Gas •Electrical IA/NI Hypochlonte Site 2 Backwash S rem Machine Shop Vehicle Ys Sludge Blending System Natural Gas •Mechanics Shop Gravity System Fire/Life/Safety/Security A/N Basins •Firers •Propane MRC-Maintenance Bates •HVAC Aeration System Polymer Alum System Reliability Center •Interior Finishes Selector Operations •Plumbing Safety •Roofing Sludge Dewa}ering Buchanan North Martinez Campus •structural UV Building •cake Pumps Annex Landscaped car •electrical =Distn�bufioh •Centrifuge Power Distribution Non LandscapedArea •Fire/Life/Safety/Security ture •Centrifuge Polymer Emergency SWGR Roads&Parking Secondary Clarifiers •HvAc Building •Clarifiers Interior Finishes Feetlers IT Buchanan South Security/Fencing/GatesMaed Liquor System Plumbin •MCC 1 •Hardware Sfortn Drains g MCC 22 Plant Control System CSOD •Street Lighting •RAS/WAS System •Roofing MCC 32 •Spam system structural Incineration Mcc 42 Co nolsy�teman Clyde •Ash MCC 43 Software Pre-Aeration Filter Plant Building •Furnaces MCC 48 Flee}Sho •Gnf System Electrical Scrubbing MCC 61 p •Pre-Aerpron Basin Fire/Life/Safety/Security MCC 81 Concord Industrial •HVAC Sub 16 Final Effluent(FE( •Interior Finishes •Sob v Plant Site Odor COntr0I FE Pumping Plumbing Sub 33 •Secondary Foggers FE Structures Roofing Odor Control •Sub 34 HHW Landscaped Nea •OuHall Structural DAF Biofilter Sub 4E Fairview Non-Landscaped Area Chemical Feed Packed Towers Sub 52 Railroad Building Sub 73 Roads&Parking •Sub 81 •Security/Fencing/Gates •ElecMcal •Su b 82 HOB •storm Drains •Fire/Life/Safety/security Flush Kleen •HVAC Sheet Lighting 'PIMP&Blower 'Sludge •runnels •In nor Finishes P ge Control •Plumbing Building Building •Roofing •Electrical •ElecMcal Lower Orindo Laboratory •Structural •Elevator Fire/Life/Safety/Security, •Fire/Ufe/Safety/Security HVAC •HVAC Inferior Finishes Steam Generation •Interior Finishes Plumbing Aux Boilers Maltby • s}/BG Plumbing Roofing Cogen Bailer MRC/Elec/In Roofing •Heat Recovery Steam Piping •Steam Turbines Martinez Primary Waste Hear Boilers Sedimentation MSB/Meth •Primary Scum Sludge LOadout •Primary sedimentation Building Moraga Basin •Electrical •Primary sludge Fire/Life/Safety/Security POB •HVAC Service Air •Inferior Finishes Dedicated Service Air OTlnda Crossroads •Plumbing •Plant-Wde Service Air Roofing Odor Control Water Systems San Ramon •Grit Bioflter •Headworks Foggers $OIdS COndl}IODIn 1 W-Domestic Water •Headworks Quads Building g w terUtility R cycled Water •Primary Covert ElecMcal 3WHP-Water High Sleepy Hollow •Primary Foggers Elevator Pressure •Fire/Life/Safety/Security 3WLP/F-Water Low •HVAC Pressure/Final Effluent Inferior Finishes • Via Roble Plumbing Roofing Wagner Acronyms: riv e) I • Final Effluent(FE) • Return Activated Sludge(RAS) Wilder Lower(private( • Waste Activated Sludge(WAS) • Ultraviolet(UV) Wilder Upper(p vate( • Recycled Water(ReW) • Utility Water(UW) • Dissolved Air Flotation(DAF) • Dissolved Air Flotation Thickener(DAFT) • Emergency Sludge Storage Tank(ESST) Switchgear(SWGR) Figure 22:Process Hierarchy • Motor Control Center(MCC) • Substation(Sub) • Auxiliary(Aux) • Water(W) • Headquarters Office Building(HOB) • Collection System Operations Division(CSOD) • Maintenance Reliability Center,Electrical,Instrument,Buildings&Grounds(MRC/Elec/Inst/BG) • Material Services Building/Mechanic Maintenance Building (MSB/Meth) • Plant Operations Building(POB) 59 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 295 of 461 Page 62 of 195 ............................................................................................................................................................................................................................................................................................................................................... I. GEOGRAPHICAL RELATIONSHIPS In the TPFleet geodatabase, parent IDs are assigned to assets to create a geographical relationship. The assignments allow staff to find related assets to assist in their work planning, e.g., a valve on a blower. .. Filter(1) ....� 71183 Inlet Filter,Ash Vac Blower 1 p.. Valve(3) ....� 71230 Iso Valve Ash Blower 1 Fit 1 ....� 71231 Iso Valve Ash Blower 1 Fit 2 ....� 71232 Iso Valve Ash Blower 1 Supervae ....�] 711:54 Ash Var Blower 2 ....�] 71176 Ash VaD Blower 3 ....�] 731342 Dry Poly Unit 1 Blower ....�] 73d52 Dry Poly U nit 2 Blower ....�] 74186 Ash Dust Bin Coll Blower Figure 23:Geographical Relationship for the Ash Vac Blower 1 ............................................................................................................................................................................................................................................................................................................................................... II. ASSET ATTRIBUTES For the TPFleet geodatabase, the asset management data includes: • Condition Score • Condition Date • Last Overhaul Date • Purchase Date • Warranty Date • Cost Center • Process • Subprocess • System • COF (Consequence of Failure) • POF (Probability of failure) • BRE (Business Risk Exposure) (product of COF and POF) • Asset ID (Identification) • Asset Position Number • Parent ID G. DATA MANAGEMENT STRATEGIES The way staff manages spatial data elements and layers are: acquiring good data; storing the data in the enterprise relational databases, integrating with business systems whenever possible; using a GIS-centric approach; sharing the data with Central San staff via user-friendly web and mobile applications; and ensuring the health of systems with analytics. The following provides more details about how staff accomplishes this: Good Data: 60 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 296 of 461 Page 63 of 195 • Attempt to achieve high data accuracy with position and attribute consistency • Ensure sewer and water network topology and connectivity • Use Esri's Data Reviewer to QA/QC multi-user edits • Use Attribute Assistant and rules and domains to ensure data consistency • Use land surveys, high-resolution aerial photos derived from photogrammetric methods, and Engineering drawings whenever possible • Allow maintenance planners to edit maintenance data to ensure up-to-date information • Developed and implemented an asset hierarchy and data standards • Update spatial and attribute data when there are changes and construction improvements • Developed a user feedback protocol via web applications or email Enterprise relational database: • Use relational enterprise database with versioning, multi-user editing, and editing privileges • Created an asset registry for the horizontal and vertical assets following developed asset hierarchy and asset standards • Expand spatial data with related tabular data Integration with business systems: • Consolidated work management and migrated to a GIS-centric, vendor-supported solution that allows integration with GIS and other programs or tools • Integrated with GIS-centric CCTV inspection platform • Implemented GIS-centric utility location (Underground Service Alert/811) ticket handling with new positive response requirements • Integrated with ERP Permit Module • Providing data and support for modeling risk-based analysis and hydraulics capacity modeling Sharing Data: • Create and publish layers for various applications to consume • Developed a user-friendly web application called GeoPortal to empower staff to search for assets and view asset management documents, reports, record drawings, permits, easements, maintenance, and CCTV inspection history related to an asset • Developed custom applications such as the CSO web application to optimize and schedule cleaning based on location and area • Developed a web-based spare parts management tool • Deployed mobile applications for field crews and technicians System Health: • Ensure system uptime using a combination of virtual machine backups (to onsite, offsite, and air-gapped media), storage snapshots and data replication to the backup data center and an out-of-region data center, and virtual host clustering • Deployed analytics tools to monitor the health of systems to ensure published layers have consistent uptime • Ensure published layers perform efficiently, speedily, and accessibly when used in applications 61 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 297 of 461 Page 64 of 195 • Shutdown and startup of virtual machines systematically (based on interdependence) H. COMPUTERIZED MAINTENANCE MANAGEMENT SYSTEM All assets are maintained through Cityworks, including but not limited to the following: • Wastewater and recycled water treatment facilities • Pumping Stations • Collection System • Recycled water distribution • Fleet and heavy equipment • Laboratory and shop equipment • Buildings and facilities Cityworks provides easy access to a map to quickly show staff the location of the asset that requires maintenance. Work orders are configured with the required fields highlighted, drop-down lists, attachments, and asset details. The system has hyperlinks that make navigation much easier than the previous systems to access related work orders, assets, inspections, service requests, or work history. Saamh.. j M0, ®Emtil Pdnt V Sa aI L✓Cla ®I M. j 1 ^ '2 n e Qsd He Q NA raa..—p— waAeeraaa.,4s9Harma q 011 Inapecheea Ed— Gelaia.CH ECKte(A1ERHW N:3 961avkha kFA. n O 3431 silo 4 Er.e,Ee.e o- � AaOe.IaIA,.aehae p p Lade,Hear.Eatead 1 E.nmmelHUa Hnnr.Entnrne -Mew„ p A.:et a::.t ulo Eoc.Eon war.ant,o.te c" i u m Emcee uesemlm:uee<c�ln lsaee la. aa -arv<�.nda.relm.mon.vl nne.r.al.nnn. � mM '" OW® �IU I OO o 3fi0 a,a s�a-lee o m E� Submd Te. rraon�v o Aaa ❑ w TY Hesc 0- Estibld En4ryTYPe 26 Prod sm. _ Am Pro—d Ee. H ❑ 6s6 wsp Nc ssewavmmAl L Pm eHAvm HAw CHEEKCH 9d6sa J� ea,rpl.ae.wo � C—� N J Sass Cbsal Hv.Genk, n G HI6 4 E,.aaaarrPa:H�,en1� Aen.l 1zrerzo169.9s AInH� � anvvsA,pi i Inahr,v -Add vaving-d work order it needed, ydg3--------1]ft v Add camems +pJd atlncdmenb._ ®Hemore allalls,Jlmen[s -- ela, pag and Ompfiles M1ere to attacM1tdem. Figure 24:CMMS Screenshot of a Work Order and Associated Map 62 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 298 of 461 Page 65 of 195 Additionally, inboxes are set up and customized by work groups, as shown in Figure 25, or by staff, depending on what they need to see regularly. Clityworks- q seam_ 1 Inbox Asset No.Lookup Asset Search , Work Order , Inspection Projects Recent Actwity . Tools Carl San Mechanical } TP Maintenance Electric Shop WOs Reacts-iC rrecti. My Shop Quick hall Asset Registry Monthly Report SHOPS Assigrodlln Mechanical Shop open wo's Ciao 7, pump Type Description Status Pri Start Date Projected Finish Date Legacy ld MP TroubleshooVRepair INITIATED 1CM21202007.O6AM 05l14202103.00 PM 77106 Recycle Press Pump 36 NorthILER TmubleshooVRepalr INITIATEN 10r05202007:20 PM 09f3020210720 PM 73132 Furnace2 We[Scmbber MP PM Applied H2O Pump 5 Month INITIATED 10H4202012.00 PM 11113202012:00 PM 42102 Applied H2o Pump 2 MP PMAL lied H2O Pump 5 Month INITIATED 10H4202012.00 PM 11113202012:00 PM 42101 Aplied HI Pump 1 ❑ 434153 TPVALVE PM Rotary Air Lock 6 Month INITIATED 10202020 07'40 PM 11#192020 07'40 PM 74197 RAL,Supervac Fitter Receiver ❑498740 TPMOTOR TmubleshooVRepalr ON HOLD 1023202011'.31 AM 07f02202103'no AM 72123 Deaeralor Supply Pump ❑ 498223 TPVALVE TroubleshooVRepair INITIATED 1026202002 32AM 1N14202002'32 PM 36230 UV 62 C3 Inlet Gate-stem cover replacement ❑ 379925 TPROILER PM Tumaraund WHB Mechanical INITIATED 1027202012:OOAM 111012021120C AM 74125 WH62 ❑ 496670 TPPUMP TroubleshooVRepair INITIATED 1028202009.27 AM 11H11202009.27AM 51111 Aer Unit2Condsr Vac Pump ❑499548 TPVALVE TmubleshooVRepalr ON HOLD 10202020 01.12 PM 082712021 01:00 PM 75508 WH61 Non Return Steam Valve ❑ 499553 TPPIPINGSYSTEM TmubleshooVRepalr INITIATED 11J021202012.00 AM 0528202103:00 PM 52121 Process Cond Return Pump 2 ❑ 500567 TPPIPINGSYSTEM TmubleshooVRepair INITIATED 11J07#2020 09 49 AM 1121202009'49 AM 974093WLP System Piping ❑ 500579 TPPIPINGSYSTEM TroubleshooVRepair INITIATED 11l031202010'.34 AM 112220201034AM 974093WLP System Piping ❑ 500611 TPPIPINGSYSTEM TmubleshooVRepalr INITIATED 11l0920200245 PM 112320200245 PM 97415 Steam System Piping ❑ 500626 TPPUMP TroubleshooVRepair INITIATED 111101202003.14AM 1124202003.14AM 31209AeFTunnel Sump Pump ❑ 5mi TPPUMP TmubleshooVRepalr INITIATED 11l10202012.00 PM 112420201200 PM 31210AWTunnel Sump Pump ❑ 500634 TPPUMP TroubleshooVRepair INITIATED 11l1720200849AM 12f0120200346AM 521125tandby Lg Pump1 ❑ 501416 TPPUMP TmubleshooVRepalr ON HOLD 11l19/2020 11.40 AM 05282021 03:00 PM 78031 Lime Slurry Recirc Pump 1 ❑ 500286 TPPUMP PM EoileF Feed Pump 1 Ye?, INITIATED 11201202012.00 AM 01 f04202112:00 AM 71152 Boll e F Feed Pump 2 East Figure 25:CMMS Screenshot of Workgroup Inbox Field staff are now "mobile," using iPads from the field to access and complete their daily work. In addition, each user can see their assigned work orders on a map, as shown in Figure 26. This mobile access allows them to plan their work for the day. With the previous system, CSOD staff would spend several hours at the beginning of each month highlighting maps corresponding to the work orders they were assigned to route their work for the month. Previously, CSOD staff manually entered requests for CCTV line acceptance into their software systems. Staff making those requests are trained to enter them directly into the system and attach the correct asset(s) to the work order. Previously, Development Services and Capital Projects staff would route a physical "green card" with the generic project IDs for the assets. When video captured, these generic project IDs were used, causing office staff to manually correct the asset IDs to match the GIS so that the inspection data would link. That manual correction was eliminated, and the assets are entered correctly at the start, significantly reducing potential errors or disconnected inspections in the system. As an additional benefit, the requester can easily monitor the progress of their request and view the results immediately upon inspection completion. 63 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 299 of 461 Page 66 of 195 Legend Selection Navigation Bookmarks Base Maps Routing Measure Locate Redline Create Achvdy Layer Search Heat Maps Data View 1 1 1 1 1 1 Legend 6 1 -Walsh Assigned ❑ 41 45 1 I ❑ l O O 1 -My Overdue WO ❑ 1 1 t -My Assigned WO For Thr p � — O r 1 1 r Recycled Water Distribution o I 1 T 1 Treatment Plant Features o 1 r Sewer Features o � mllealon t 1 dgec�o r Administrative Areas m �amP� S v eome:as 1 r Reference Data m S` r , , , r' 1 71 1 , r r , , r , Figure 26:CMMS Screenshof of Mapped Work Orders In addition, new workgroups are using the CMMS to manage their assets, specifically the Laboratory and Control Systems. The asset registry was reviewed and updated with the workgroups, work order templates created, and recurring work orders triggered. The CMMS system was configured to easily access asset and work order history, enhanced planning, scheduling, work execution, and continuous improvement. Other significant improvements are implemented as follows: • Enhanced inspections • Readings, comments, and details on assets • Fault, Action, and Cause codes • Optimization efforts tracking using Reliability Centered Maintenance (RCM), Experienced Centered Maintenance (ECM), and Planned Maintenance Optimization (PMO) o QA/QC of work orders o Return by Supervisor o Planner update o GIS update o Safety, Quality, and Effectiveness o Enhanced reporting • PM triggered by run times from SCADA • Use of mobile devices • Regulatory compliance monitoring • Ability to link documents and other files to work orders • Equipment under warranty notice on work orders • Spare parts linked to the warehouse • Automatic link to materials drawn from the warehouse against a work order 64 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 300 of 461 Page 67 of 195 • Safety work order tracking • External vendor cost tracking Staff may use the following email, cityworks@centralsan.org, for support. I. CLOSED-CIRCUIT TELEVISION In 2016, staff implemented ITpipes to replace the previous CCTV software platforms, WinCan and Granite XP. The ITpipes software integrates with the GIS and Cityworks CMMS platforms. ITpipes includes a web plug-in, allowing GIS staff to link the data to GeoPortal, including the videos, which was not possible before. J. AUTOMATIC VEHICLE LOCATING In 2017, CSOD piloted the Verizon Network fleet for automatic vehicle location and global positioning system. The system monitors vehicle diagnostic issues to streamline preventive services, increase productivity, and efficiently dispatch crews for emergency response. K. UNDERGROUND SERVICE ALERT LOCATE TICKETS In late 2017, CSOD implemented Dig-Smart Fusion to automate Underground Service Alert (USA) ticket management. The new software integrates with Cityworks to generate Service Requests after Supervisor review, eliminating the manual creation of work orders that staff previously did. The software links directly to published map services. L. SPARE PARTS MANAGEMENT APPLICATION The Treatment Plant's previous CMMS system, Mainsaver, had a module for associating and tracking spare parts for its managed assets. Cityworks does not have this vital functionality, so in 2018, staff developed a Spare Parts Management Application to address this issue. They cleaned up and imported the previous spare parts information from Mainsaver and worked with the Warehouse staff and Maintenance Planners to refine the existing lists and add additional lists for newer assets. The application shows Maintenance staff if spare parts are available before beginning any work, without needing to ask the Warehouse. It also helps the Warehouse effectively manage spare parts inventory for PMs. When a spare part is available, staff can easily respond to work orders on PM cycles and minimize production downtime. This application is accessible from within the Cityworks application for staff in Plant Maintenance and Materials Services (aka. the Warehouse). From the main Spare Parts Management screen, a user can look up the parts for a specific asset (pump, blower, mixer, etc.), view a summary of all spare parts lists, or see which assets (if any) use a specific warehouse stock item. A spare parts list can be found by searching using the equipment or asset position, a portion of the asset's name, or the manufacturer and model number (if known). The search can often return several matching assets, so the user must select the one they are interested in. 65 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 301 of 461 Page 68 of 195 Spare Paris Management Sj—Perta Mene9—na Parts LIu bl Ault 25110 Wki-t►ueq I ffl&Mp1M 12MlFMI Q r.rwwwrr..r � � rr� 7aq nWegrRMa.rAwbm,epm w[a�oce n.r•bwex aom ry r.A 1.qx+ nq.e ..p.e fu.wlfarw Nwn 1`.wM N.ww.. ow..n Ge.r!'h a1.wr W...w. w.. onnwar w .r. urru ew... rw a.+^ro C.wwe ov om enm ussuwmx.e ue.emm .0 .m .re im ao1 ourru•.wr. oaoo,orooe aw naetc..q*nros eex,aoen 1� .m om o00 cry e?srm.srnw eesor omr owar.rwano wslmm :tu o0o xoo :oo ruooawme �GW(anG WEUER .crrmm .se .m .m im pv.v�9x qa! vesaw um.uer,urxin. xuwo xu owoexa arw as>eueuwuu uc.aasa. xu xw Figure 27:Spare Parts Search The spare parts list is loaded, combining information from the GIS asset registry, the spare parts database, and Central San's Enterprise Resource Planning (ERP) system (formerly Sungard, now Oracle). It shows the warehouse stock number, some basic information about each part, the precise warehouse shelf where the item can be found, how many parts are required to repair or overhaul the asset, and the current quantities on hand. This list gives Materials Control and Plant Maintenance staff quick access to the exact parts needed to repair or maintain the assets. Maintenance Planners update the spare parts database, and the Materials Services Supervisor updates the warehouse portion of the Oracle ERP system. The "Where is this Warehouse Stock Item Used?" tool allows Materials Services staff to see what (if any) assets might use a specific spare part stored in the warehouse. If warehouse stock items are not used for an extended time, this tool can be used to determine if any assets are still in service that might need that part. Spare Parts Management Where is this Warehouse Stock Item Used? Commodity 034 Descriptic, KIT MECH SEAL REPAIR FOR 5.815"SPLIT JOHN CRANE i sub Commodity 003 Reference No. It.. 00004 Manu.Part No. Location MC-la-20-05 Minimum Quantity 1 Maximum Quantity 1 Cancel Figure 28:Spare Parts Storage Locator 66 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 302 of 461 Page 69 of 195 M. RELIABILITY ENGINEERING TOOL Another important addition to Central San's GeoPortal was the Reliability Engineering Report. The tool classifies assets in a process hierarchy, making searching, querying, and analyzing assets faster. With the tool, a user can see spare assets or parts attached to an asset. The user can also view which assets have PM and PdM (e.g., vibration and temperature analyses), work order cycles, and which assets have a maintenance program assessment, such as RCA (Root Cause Analysis) or RCM (Reliability Centered Maintenance) analysis. It also shows the following attributes for performing risk analysis, which are continually updated based on asset-based condition assessments: • COF • POF • BRE fir✓— • 4 J I N T1 ,� r l ,I ­1Yurrar 0"Wri7.cwfk M+ P_d hh W WVMB.• -. JiY Fa4 l NO •wM l 4Nt i+syt P4n b o Sor rd w%c [aM r—. r,.a.. 5a— Cb. PALkH. ca.gr.. t..ft.W :cr,IG,,.d e'ar. oow c..v rr ,,.�. .. .. o a 0 O a+,a �.w z A O r~+r riq G ism. ��yy y,� 'ISh 7 i S JaMm'.�ri.,+ S'H y r—r+f. cw".K. •J �/ Figure 29:Reliability Engineering Report Tool N. DOCUMENT MANAGEMENT SYSTEM Originally implemented for the Administration Department to store Board documents, Laserfiche stores various documents, many of which are related to asset management. Many of these documents are linked to capital projects, assets, or parcels through their metadata and can be retrieved through GeoPortal. The GIS staff uses Adobe ColdFusion to create reports accessible through GeoPortal that query the documents based on the feature selected in the map. For example, a maintenance worker can select a pipe from the map and retrieve the record drawing from when it was installed. 67 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 303 of 461 Page 70 of 195 Various templates were developed in Laserfiche to store the metadata for different documents. The templates most used and accessible through GeoPortal are: • As-Is Drawings • Assessor Maps • Asset Management (Condition Assessment, O&M Manual, Permit, Shop Drawing, Technical Report, and Schematic) • Development • Right of Way (Grant Deeds, Real Property Agreements, etc.) • Permits • Record Drawings SOPS are also stored in Laserfiche with their template but are not linked to GeoPortal. For more information, see Section 7. O. MODELING SOFTWARE To support operations, maintenance, and engineering activities, the Planning Group has hydraulic and process models for the collection system, treatment plant, and recycled water distribution system. ............................................................................................................................................................................................................................................................................................................................................... I. HYDRAULIC MODEL FOR THE COLLECTION SYSTEM A hydraulic model of the collection system is required by regulations and is used to identify capacity needs. Planning staff and their modeling support consultant use the model to evaluate hydraulic conditions and capacity constraints for sewer renovation projects, collection system planning and sewer alternatives, special discharge requests, new and re-development, and other special requests to better understand system performance and conditions. Central San replaced an old steady-state model (ArcSnap) with a new, vendor supported, dynamic model (Innovyze's Infoworks ICM) as part of the CWMP (Carollo Engineers, 2017) and also conducted flow monitoring to help calibrate the model. Similar to other agencies, this model encompasses the entire service area but includes dynamic modeling functionality and calibration of primarily the trunks and interceptors. Planning staff anticipate expanding it to smaller mains for priority development areas. For additional information, please see: • TM CS-6, Flow Monitoring • TM CS-7, Parcel Loading Process • TM CS-8, Model Network Development • TM CS-9, Model Calibration • TM CS-10, Model Implementation • TM CS-1 1, Future Land Use and Model Loads • TM CS-13, Long-Term Flow Monitoring • TM CS-20, Capacity Analysis Results ............................................................................................................................................................................................................................................................................................................................................... Il. HYDRAULIC MODEL FOR THE RECYCLED WATER DISTRIBUTION SYSTEM In July 2018, West Yost & Associates developed a new hydraulic model for the recycled water distribution system using InfoWater software, which included a pressure transient analysis using InfoSurge. 68 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 304 of 461 Page 71 of 195 ............................................................................................................................................................................................................................................................................................................................................... III. HYDRAULIC MODEL FOR THE TREATMENT PLANT For the treatment plant, Carollo Engineers developed an Excel-based model called Hydraulix in 2010 to calculate the hydraulic profile through the plant unit processes, connecting pipes and channels. The model estimated the allowable peak flow that could pass through the plant without overtopping the hydraulic structures (tanks and channels) while maintaining a minimum freeboard. The model was updated as part of the CWMP (Carollo Engineers, 2017), described in TM TP-7, Treatment Process Performance, and Capacity Evaluation. DP 7349-Steam and Aeration Blower Systems Renovations Project used the hydraulics software package, Visual Hydraulics to understand the hydraulic capacity bottlenecks through secondary and disinfection processes and verify the CWMP outcome that two new secondary clarifiers were needed to address hydraulic constraints (HDR, 2021). Note: a red exclamation point on an individual hydraulic element reflects a situation where the hydraulic capacity for that element is exceeded at that defined flow. Sta¢ 114.1E Il-03E SSl9B 1]AJ4 11-0A5 11-0SE 11521 IIEJ9 ]16J9 116J9 ]ice 111AE 11,.1, I S�aengrmlc wdme ppeomnected toff oouEle Conduit-Foot GbmeM�g PmtAneion 5hsp fndEHlumt Cmncrtim 96�dFff 72 Fdiff ffGmnd Broad CresiN Wa Pmtlmndn VV VVlandn Gale �....... dereem PS VVlEGmnd M.fi k [rcud Vler mad raatreaem l 7 Gmnd 118A1 118A3. n7m 117M 117J4 MA 117.79 111J1 Il]JI 11159 11719 I111B n7] IIl2; At No E01umt51uioe S,k Cbs Chmnd S'kO Gmnd Qauhk Cked Nonh 51 l,ktG- ' Gate 2 1 Gmnd Seoondsy FffLmr Sxmdsy FAlum Pre-VV Grind VV Grind Yffet G,UV GmndM VV Grind VVGmnd llude� Post VV Grind VV Wa \ Gmnd Ou G_ Lora Gne 1 11&16 110b9 1]1IB 111.43 / ♦ W �\ No and 9outM1 Spit NoM Y t EHlumt NoM 5[I launder V-NoteM1 Effluent Wei NwM SC 1 iiffumt � ]31J4 11836 118A] 11939 119J4 ]39J4 119EF TrouEN Or^P INoMI PpeFkt I bu�N SE Lae uth SC ElNueni Pipe luncim Strunuxl butN YEIPIumiPye both SC ElPlumt butM1 E01umt51uix osuUt SC IEAIIumt 111]] 1111] 111)6 111)1 1 2 Landon G_ Tta 00— NwN NML Gmnd No NMLk G_ Na NMLCanduit No SC I loll M1 112.77 12014 119.14 119b5 119.4 11L45 I3&]6 1]5.19 I SoutM1 MLtik,Lae butF SML Conduit SautF SClidlumt SautF SCIkllumt V�lotdi EAllumiWa Soufi SCllncd AtAt Pgn Ppeiikt 1swmY `-0�ml' I lA 5v�t9isp IA Conn«trgPpe IA[Rlumi Wer 121J4 111A1 12321 12314 GciW Ws OF ( At ( ­t ( —4 butM1 NML Gann, 2A5iragh19 2RCann_,P'__• 1A1M121'v:_� Crested Wdr Figure 30:Visual Hydraulics Software of Central San Hydraulic Capacity Simulation ............................................................................................................................................................................................................................................................................................................................................... IV. PROCESS MODEL Central San uses BioWin, a proprietary model simulating biological processes, to model the treatment process. It was first implemented and calibrated for the Treatment Plant Planning, DP 7301, and Treatment Plant Master Plan, DP 7343 to evaluate potential nutrient reduction alternatives. BioWin was used for the CWMP (Carollo Engineers, 2017) to estimate the secondary treatment process capacities and solids processes for current and projected flows and loads, described in 69 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 305 of 461 Page 72 of 195 TM TP-7, Treatment Process Performance, and Capacity Evaluation. DP 7349-Steam and Aeration Blower Systems Renovations Project calibrated the 2017 model in 2022 and considered diurnal variability. This model is recommended as a tool for sizing and potentially designing future facilities, performing alternative evaluations, and developing facility needs and cost estimates (HDR, 2022). 70 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 306 of 461 Preeeretiprd Spr her Heedwprks PYerlsry craler OCU Te AlryiBs$hlre OCU Td AliHpipNM fJ - .......................... r�Fyym Inke"Inbona ;--- --# •� ------ r 1 Elll P1ors ------Bypass I Sodium i111pSUIretE iron C.11-hen Sysrm 4 Swdum I FlUding Hypodilorne B.A4 p erbilm 1 i Basins Drsinhac*[o Bsdcup 3yo -.-,-._._.-,-, n I r r Y i' Mertiner Tnrrl [ ' A 1 ! ! Eme Sodium Prrn ! oudadB a Se dart_i I uen[ ! i .Y I r •-•-FinalEriluent WaftrtCreek Stu.to D�xs+an St &umnr9 i SBcandary Eflkjmt 6AFTs - 3lruoivea - r i l'►ridrening; r r ; System 'Pdmary EMc+errcu Anaerobic NN Seoorldwr Erfiuen4 to Bar Innuanl 'Slandley E1Auer11 S010dor Tarklm Clartliers iiDFi­v r&}'seScreens Pum ! IPumpa w HightHill flew �qMYav11y11,41 eptors Nteather k+let rj� i5dw7sn1Aerated Gnt seal tanon(P—Aeralicn) sln} Air fmm BkmersEmergency Tenke •.� isinrectte Qrayv Creek Hydrogen Screening& MS2 M$3 C Holding Holding ea EfBuBM &uisu-Bay Peranide �lVasrler7 Air Irom ar Sodium CwnRactnr Blehy3 Basis Basirw Rumps Hypochlonle Grit Maw.? 3 fuS GA.N—dd) CIaBaWer pump Alum $odium Q (As Ndededl 11YpoL1110n1E Fl j WASP alMadla F and b Pumps FIu s Sewer Screenings (315110 Lerlalill F9relf l0 Ls ntl811 k r 0 m SidiPrDge, Premg Applied +I Firm Buddi Water Clan oll ......Control Bu..... .................... Pump. SGru44ar 1p .................................................................. :.. water t332 tteeycied Inlluant hem 0 Q Recycled walar users G.U.6on System WAS N+ater Fumes (Vwln Trunk) oAFrs 'Npa1B Rner Wade Badnrash $MkwAgh lip Backwash Pump. M1t&anC21Jn0 Tank _ r� Cenlroke I.Auvrial Wiugr Pumps Not Used] Frum LE9END Cenh luges To 10 d undernow Na 2Ur�ter DAFT-Clssemed All Thlckener d UGily 4vater Syetem jUndk-.HID. 'I Pumps WAS-Waste Acliosl[ed Sludge WAS-Rau,Activated SludgeA1N Atr+31i0nfNRrifcab6rf Q Primary W Uluseiolel Prima Pr' Sludge 7 1�Y m sludge OCU-Odor C9rmol Urim Sludge Sludge el"di%Te k — -UquldFlinifto r'ulr"Vs °nridet, CENTRAL SAN ----A.�oulwrP.ln Main Stream Sampling Location PROCESS FLOW SCHEMATIC Opkwnal rlp ping LIQUIDS HANDLING -----•WQl VMA1her 6ypas3ing IRSReturn Strearn Sampling location Figure 31:Sampling Locations on Process Flow Diagram 71 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 307 of 461 Page 74 of 195 1 . RISK DEFINITION In asset management, risk can be described as "the effect of uncertainty on the delivery of the asset management objectives." This means understanding the assets' behaviors in an unexpected manner or one that is outside of the design specification. Understanding which assets are most critical and their condition allow staff to focus on maintenance, renewal, and rehabilitation investments. There are two components of risk, which are the: • Likelihood of an event happening, and • Consequences of the event on the organization. Risk is generally quantified by multiplying these two factors and is usually expressed as a numerical score. Understanding risks help Central San reduce financial and performance losses, and using an informed, risk-based approach to managing assets leads to higher operational reliability and consistency of service. Asset risk management consist of these activities: • Identifying risks, • Quantifying them, and • Controlling them. 2. IDENTIFYING RISKS - CONDITION ASSESSMENT Condition Assessments can identify risk at any time during an asset's life. A condition assessment is the systematic review of the physical capability of an asset to perform its function in its operating environment. Condition assessments are an integral step in determining an asset's remaining physical life and establishing the LOF, which is a critical component of the BRE calculation. In 2020, Central San partnered with HDR Engineering Inc. (HDR) to complete the Steam Project, which generally consisted of data collection, condition assessment, testing, business case evaluations, and pre-design to address the following objectives: • Employee safety • Reliability of operation • Meeting current and future demands • Reliability of permit compliance • Reduction of operation and maintenance cost The condition assessment results would be key to estimating the RUL of critical equipment, developing lifecycle costs, and establishing a baseline for use in evaluating alternatives. This project marked an important milestone in Central San's reliability journey; using condition assessment results to guide construction projects of this scale had never been done before, and it would serve as a model for future projects. The levels of condition assessment were defined as follows: • Level 1 - Review of Age and Expected Life • Level 2-Visual Field Inspection • Level 3- Performance Testing • Level 4- Non-Destructive Testing • Level 5- Forensic Testing (Destructive Testing) 72 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 308 of 461 Page 75 of 195 40 • Level 5 Forensic • Level 4 Non- Testing (Destructive Destructive Testing) Testing Level 3 Performance Testing Level 2 Visual Field Inspection • Level 1 Review of Age and Expected Life Figure 32:Condition Assessment Levels The scope of the project was expansive, with condition assessments performed on the multiple-hearth furnace waste heat boilers, cogeneration boiler, auxiliary boilers, deaerator, purification system, boiler feed pumps, steam and condensate piping, chemical feed systems, blower turbine drives, other steam users, sampling and analysis systems, and associated electrical systems. The Steam-System condition assessments were generally defined as Level 1 or 2 assessments, but in some cases as Level 3 or 4 assessments, depending on the system. The condition assessment process consisted of 3 phases: Planning and Initial Evaluation, Inspection, and Engineering Analysis. 73 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 309 of 461 Page 76 of 195 --------------------- +fir--------------------- %� -------- + t i 4 1Engineering1 l 1 1 L • 1 l i 1 1 1 1 1L J1 ' 1 1 1 1 1 I 1 1 1 1 1 I 1 1 1 iVisual conditionassessment of the asset I j j I I j I 1 j I 1 I I I I 1 1Recording available data I I 1 I Requesti Review I Asset 1 I 1 1 i I 1 I 1 1 1 1 Establish ; I 1 IScodng System I 1 j ; Noting observations ; I 1 I 1 1 I Esfimate Remaining Devel1 � 1 1 41 1 4 I'tiflb6 Life ' I 1 I 1 E • t Assessment Plans 1 ± i ti 1 1 ti 4 1 fry# # --------------- + --------------------- ~� �------------------------- Figure 33:Condition Assessment Phases 3. QUANTIFYING THE RISK r A. PROBABILITY OF FAILURE, CONSEQUENCE OF FAILURE, AND BUSINESS RISK EXPOSURE To evaluate the risk, staff needs to quantify the probability and consequence of an event occurring. As part of the Planning and Initial Evaluation phase, a criticality scoring system was developed and defined by Central San staff with the following attributes: • POF (Probability of Failure): defined as the estimate of the relative time to failure based on condition, performance, operational function, and maintainability. POF is also known as the LOF. • COF (Consequence of Failure): defined as the impact due to asset failure and considers operational, social, regulatory, economic, environmental, and other factors. • BRIE (Business Risk Exposure): calculated as the product of the POF and COF and defined as an estimate of the relative risk that an asset presents to Central San's ability to meet commitments. Higher BRIE values represent a more significant risk. BREs lend themselves to grouping/ranking, which will evaluate options. The current Central San system assigns a condition score of 20 to refurbished assets. The HDR team will give a score of 20 to observed refurbished assets. Condition Score is on a 10-100 scale, while the POF is on a 1-10 scale. 74 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 310 of 461 Page 77 of 195 Table 6:Asset Conditioning Scoring Central San HDR POF Description Percent of Maintenance Ops & ConditionOriginal Maintenance • - Life Condition Score 10 1 1 New or 100% Normal excellent preventative condition Maintenance 30 2 3 Minor defects 75% Normal only preventative Maintenance, minor Corrective maintenance 50 3 5 Moderate 50% Normal deterioration preventative Maintenance, major Corrective maintenance 80 4 8 Significant 25% Rehabilitation, if deterioration possible 100 5 10 Virtually 1% Replace unserviceable B. CONSEQUENCE OF FAILURE SCORING Central San's Asset Management Team has assigned the COF in collaboration with appropriate stakeholders. The COF is primarily driven by operation criticality to meet Central San's safety, operational reliability, permit compliance, and cost-efficiency goals. The original values for COF were developed in workshops in 2013 and then approved by POD and Plant Maintenance in 2013 and 2014, respectively. The Central San COF scoring system is identified below in Table 7. Table 7:Consequence of Failure Scoring COF Score Description 10 Catastrophic 9 Critical 8 Significant 7 Major 6 Moderate (high) 75 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 311 of 461 Page 78 of 195 5 Moderate (low) 4 Minor 3 Minimal 2 Negligible 1 Insignificant C. BUSINESS RISK EXPOSURE HEAT MAP The Business Risk Exposure (BRE) is a measure used to estimate the relative risk that an asset presents to Central San's ability to meet commitments. Understanding which assets are most critical and their condition allows staff to focus on maintenance, renewal, and rehabilitation investments. The COF and POF are quantified on a 1 to 10 scale. The BRE is a 1 to 100 scale, where lower values require little to no near-term investment, and higher values require significant near- term investment. By assessing the BRE, Central San can focus the investment on renewal and condition assessment on assets that pose the most considerable risk. The BRE parameter can produce a "heat map" of risk associated with assets, as shown in Figure 34. The BRE Score incorporates all information collected during the condition assessment, system reliability, and redundancy. Assets in high categories will be prioritized for near-term intervention, whereas those in low and medium categories will be monitored until their risk increases. Probability of Failure (POF) Low Med. Low Medium Med. High High High Medium High Extreme Extreme Extreme O Med. High Medium Medium High High Extreme Medium Low Medium Medium High Extreme 0 a u Med. Low Low Low Medium Medium High a a, Low Negligible Low Low Medium High U Figure 34:Heat Map of Risk as a Result of COF and POF D. AGGREGATE CONDITION ASSESSMENT SCORING HDR's condition assessment process included at least one of the following condition assessments: • Level 1 - Review of Age and Expected Life • Level 2-Visual Field Inspection • Level 3- Performance Testing 76 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 312 of 461 Page 79 of 195 • Level 4- Non-Destructive Testing • Level 5- Forensic Testing (Destructive Testing) For example, a Level 1 assessment would have a Condition Score, POF, COF, BRE, and %of Original Life Remaining based solely on the equipment's age. Similarly, a Level 2 assessment would have scores in the same categories based on a visual inspection and records review. Other assessments would likewise have their category scores. The individual scores for each level of assessment must therefore be combined in some manner to reach an aggregate score based on all the assessment levels performed. Table 8 shows the weights for each level of assessment for steam systems. Table 8:Condition Assessment Level with Aggregate Weight Condition Assessment Level Aggregate Weight Level 1 - Review of Age and Expected Life 15% Level 2-Visual Field Inspection 30% Level 3- Performance Testing 25% Level 4- Non-Destructive Testing 30% Level 5- Forensic Testing (Destructive N/A Testing) Note that the Aggregate Weight works best when an assessment level is performed. The assessment scores use a fixed value for the Level 1 (Age) condition assessments of 15%, regardless of what other assessment levels were performed. The weight of an unperformed assessment is split evenly between assessment levels that are performed. For example, the cogeneration boiler has scores for Level 1, Level 2, and Level 4. No Level 3 (Performance) assessment was performed. The aggregate score is determined by multiplying the Level 1 score by 15%, the Level 2 score by 42.5% (30% + 25%/2), and the Level 4 score by 42.5% (30% + 25%/2). Table 9 summarizes the weights associated with different condition assessment levels based on which levels were performed. Table 9:Condition Assessment Level Weight Adjustments Based on Levels Performed Levels Performed Level I Level 2 Weight Level 3 Weight Level 4 Weight Levels 1 & Level 2 15 85 0 0 Level 2 Only 0 100 0 0 Levels 1, 2, &3 15 45 40 0 Levels 1, 2, 3, & 4 15 30 25 30 Levels 1, 2, & 4 15 42.5 0 42.5 77 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 313 of 461 Page 80 of 195 The Level 1 assessment (age) impacts the overall condition of the equipment, but other assessment levels are more accurate and therefore weighted accordingly. If only a Level 1 and Level 2 assessment were performed, for example, the Level 1 weight in the aggregate score would still be 15%, and the Level 2 assessment weight on the aggregate score would be 85%. Table 10:Condition Assessment Weighting for Steam System Available Information Condition Age Only Age and Age and Age, Assessment Level M Inspection (%) Testing (%) Inspection, and Testing (%) Level 1 -Age 100 70 70 45 Level 2-Visual 30 20 Inspection Level 3 /4-Testing 30 35 Levels 1, 2, &4 15 42.5 0 42.5 Table 11 tabulates the weightings used for the various combinations of available information. Not all assessment levels are available for each item of the electrical equipment, so the weighting factors must account for all possibilities. Electrical testing, by its nature, is intended to be non- destructive; thus, levels 3 and 4 are combined into a single testing score. Table 11:Condition Assessment Weighting for Electrical Equipment Available Information Condition Age Only Age and Age and Age, Assessment Level M Inspection (%) Testing (%) Inspection, and Testing (%) Level 1 -Age 100 70 70 45 Level 2-Visual 30 20 Inspection Level 3 / 4-Testing 30 35 Total 100 100 100 100 This information was then used to guide decision making on asset replacements for the Aeration Basin Diffuser Replacement Project (DP 100019) and Steam System Renovations Project Phase 1 (DP 100032). Please see the following for the detailed condition assessments: • Condition Assessment Reports Executive Summary for Steam and Aeration Blowers Renovation Project • TM No. 09-Steam System CA Report • TM No. 10-Electrical • TM No. 1 1 -Secondary CA Report 78 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 314 of 461 Page 81 of 195 4. CONDITION ASSESSMENT PROGRAM Understanding which assets are most critical and their condition allows staff to focus on maintenance, renewal, and rehabilitation investments. Adoption and implementation of this risk-based approach are expected to result in overall cost savings to Central San by: • Prioritizing investments in the highest potential risk areas • Optimizing the maintenance costs over the lifecycle of the asset • Focusing condition monitoring and assessment on critical assets • Reducing the numbers of emergency repairs and associated clean-up or restoration if critical assets were to fail • Maintaining Central San's reputation and credibility with ratepayers, regulatory agencies, and other stakeholders A. COLLECTION SYSTEM The following sections describe the collection system's condition assessment program in more detail. ............................................................................................................................................................................................................................................................................................................................................... I. GRAVITY SYSTEM Central San applies a Level 2 assessment through a CCTV inspection program that began in 2002 (Phase 1), utilizing an outside contractor for the collection system. The majority of the collection system was inspected by 2010 at an approximately$7 million cost. In 2011, Phase II was initiated to reinspect sewers and is ongoing. Beginning in FY 2014-15, the CCTV Inspection Program is allocated through the O&M budget instead of the CIB. An analysis in 2018 indicated that at least 95% of the gravity system had been televised. Further evaluation is needed on the last 5%as there are discrepancies between the pipe and structure IDs in the CCTV and GIS datasets that could negatively impact the analysis. The primary objective of Central San's CCTV inspection program is to obtain current information on the sewers' internal condition and use the data to optimize maintenance and renovation programs. An essential aspect of this program is the immediate notification to CSOD of any problems that could cause a stoppage or overflow. A master ITpipes SQL database on the GIS servers stores the inspection results. The inspections are accessible through GeoPortal by integrating the ITpipes viewing software. The last inspection date and score are stamped as an attribute to the pipe asset to create GIS layers visible in GeoPortal. Furthermore, the InfoAsset Planner model uses the ITpipes results as input to develop a risk-based renewal program for the collection system. The CCTV score identifies the future needs of the pipe, as shown in Table 12.The total sanitary sewer overflows by city before 2003. However, in Phase 2, the inspection order followed a standard rotation around Central San's service area. 79 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 315 of 461 Page 82 of 195 Table 12.Sewer Selection Criteria Categories Category CCTV Score Defects Future Needs 1 >5,000 Extensive defects (dozens of root intrusions, multiple Evaluate for structural failures, shifted joints, holes in pipe, and renovation sags-Category 1) 2 1,500-5,000 Several defects (greater than the very few groups Continued but less than those needing replacement- observation Category 2) 3 <1,500 Very few defects (fewer than five minor defects or None, in one major defect-Category 3) excellent condition Since its inception, the CCTV program has evolved to include several additional observations with subsequent activities that include: • CSOD is immediately notified if the contractor encounters: o standing water o structural defects o significant root/debris build-up or obstructions where the inspection could not be completed o paving defects around maintenance holes o easements encroachments • Maintenance Hole Inspection: Interior and exterior of all maintenance holes (not just CCTV inspection entry points) are inspected/photographed, including the top block, frame, and cover. • Cleaning of Pipes: Pipes are only cleaned as needed to facilitate the passage of the CCTV camera. If the line cannot be cleaned sufficiently to pass a camera, CSOD is immediately notified of the incomplete run. • Laterals with Roots: Owners whose laterals have significant roots are notified, where feasible, through in-person notification if the resident is home when the crew discovers it during the inspection; if they are not home, then the homeowner is sent a form letter explaining the situation and how to address it and a picture from the CCTV inspection. When pipes are installed, Central San accepts responsibility from the contractor following a passing CCTV inspection. If the pipe fails the inspection, the contractor must repair and CSOD will repeat the inspection to verify it passes. Following the CCTV acceptance inspection, the pipe will be re-inspected when 30% of its expected service life has passed or the pipe requires more than routine cleaning (i.e., a ten-year schedule). Once the inspection cycle begins for a pipe, the inspection frequency will range from 15 years (good condition, <1500 points) to 5 years (poor condition, >5000 points). The criteria used to select sewers for the CCTV re-inspection are the most recent CCTV inspection score, the last inspection date, and current cleaning frequency (see Figure 35). The total pipe footage is selected to stay within the current FY's O&M budget. 80 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 316 of 461 See Tips EvalicGon Cbmry CCTV whew_ Pro.rz35 Ci�ryGlSwhrre= ill Lastsmrek 15oD City=TBD AND Last date 2 S vem.5 Ciil�1M Lam date 15 years Cbeate spreadsheet with: Is ID facility lD sale approx2. D, -pipedometergI«I] CbmryGAMSfvr: LFofpipeperllclexe{e.g.3 TieIrn �FfJ Last PMtempLdm=TBD months oPvrork] Tiipe.na.erial List PM cyrf S THD mD Aatit PM CleflminG 4-at P M{.Wde -Alternating Schedule 4but score tit d ate GLxto produm maps vide IISCtapriitmlorerd maps C54 provides rzleamto selected limes hiahli IA7 Carrtra=r.which inriides tftc spreadshemt amd maps Page 84of105 Elm I No Yes No ND- Yes ngvne3m:Collection System Condition Assessment Program ---------------------------------------------------------------------------------------------------------------- ||. LARGE DIAMETER GRAVITY SYSTEM The Planning Section administers the Large Diameter Pipeline Inspection Program, DP 8443, to assess the condition of |orge'diometerhonks and interceptors and confirm the timing for replacement needs. 82 Central San Asset Management Plan December 21. 2023 Regular Board Meeting Agenda Packet' Page 318of4G1 Page 85 of 195 For the large diameter pipes (>_24") in the collection system, Central San applies a Level 2 or 3 assessment depending on the risk of the asset. This condition assessment program was developed with the CWMP (Carollo Engineers, 2017) to supplement Central San's CCTV condition assessment efforts; for more detailed information about the program, see TM-CS-14, Condition and Risk Assessment, Appendix C. In the FY 2017-18 CIB, Central San included the Large Diameter Pipeline Inspection Program, DP 8443.This program is expected to be funded on 5 to 10-year cycles. The CWMP recommended optimizing expenditures by implementing a tiered inspection program based on the risk of the asset (see Figure 37). Tier 3 •Inspect every 5 years using enhanced CCTV h Risk and trunk walks (where feasible) • -Candidate for routine PM such as H2S monitoring and microbrial control treatments Tier 2 •Inspect every 7 years using multisensor CCTV and trunk walks Medium •Candidate for sonar/laser inspections to evaluate rehabilitation options or quantify (18 miles) sediment build-up Tier I -Inspect every 7 years using CCTV and trunk Low Risk walks (where feasible) (56 miles) •All small diameter sewers Figure 37: Risk Pyramid for Condition Assessment of Large Diameter Collection System Pipes A phased large-diameter inspection program is being developed under the Sewer System Infrastructure Plan starting in 2023. These were the recommended inspection schedule from the CWMP: • Inspect nine miles of high-risk large-diameter pipelines every five years • Inspect six miles per year of medium-risk large-diameter pipelines for the first three years, followed by three miles per year after the first three years (seven-year inspection frequency) • Inspect eight miles per year of low-risk large-diameter pipelines (seven-year inspection frequency) 83 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 319 of 461 Page 86 of 195 ............................................................................................................................................................................................................................................................................................................................................... III. FORCE MAIN SYSTEM The Planning Section administers the Force Main Inspection Program, DP 8444, to assess the condition of force mains and confirm the timing for replacement needs. The remaining lifespan of individual force mains is challenging to estimate without inspection-based condition assessment results. Force main failure methods include internal corrosion, external corrosion, mechanical failures due to high pressure and surge events or external loads and stresses, and force main primary material or installation defects. Recommended force main inspection methods include CCTV inspection, pressure transient monitoring, acoustic leak detection, and electromagnetic inspection. For the pressurized pipes in the collection system (i.e., force mains), Central San applies Level 2 and 3 assessments at a frequency that varies depending on the risk of the asset. This condition assessment program was developed with the CWMP (Carollo Engineers, 2017); for more detailed information about the program, see TM-CS-14, Condition and Risk Assessment, Appendix B. Tier 3 •Inspect every 10 years using CCTV and tethe High COF acoustic testing for leak detections -Monitor for pressure transients every 5 year Force . •If necessary, use electromagnetic inspections -Inspect using CCTV and tethered acoustic testing Tier 2 every 10 years Medium •Visuall inspection look for offset joints or obvious signs of internal corrosion of . - . -CCTV and acoustic inspections can be staggered every 5 years or same 10-year cycle Tier I •Inspect using CCTV and tethered acoustic testing Low COF every 5 years •Where infeasible, rely on age and maintenance Force . data Figure 38:Risk Pyramid for Condition Assessment of Force Mains The CWMP recommended (see Figure 38) optimizing expenditures by implementing a tiered inspection program based on the COF consequence of failure of the asset. This phased force main inspection program is being developed under the Sewer System Infrastructure Plan starting in 2023. These were the recommended inspection schedule from the CWMP: • FYs 2017-20, inspect force mains at Moraga, Orinda Crossroads, Lower Orinda, Bates Boulevard, and Wagner Ranch Pumping Stations • FYs 2020-27, inspect force mains at San Ramon, Martinez, Fairview, Maltby, Clyde, Concord Industrial, and Acacia Pumping Stations • Other pumping stations' force mains with lower risk will be inspected in a later phase 84 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 320 of 461 B. TREATMENT PLANT, PUMPING STATIONS, AND GENERAL IMPROVEMENTS The inspection of the Steam Project with HDR informed Capital Projects phasing. Aeration WANEV r Diffusers jI� �� Mixed Amb Liquor dt�YRjiiRu i+.8 i A Channel ig3}L� Mixing Pipes 00: Electric Blower ,= .. m s i* Clarifier Aeration Mechanical System Air u lun um��s:;` h Equipment Filters uuwiwurii 7. r, Figure 39:Inspection of Secondary Process High Value Assets and Support Equipment Informed Capital Projects Phasing 85 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 321 of 461 Waste Heat Boiler Heat Reservoir Exchanger `\ -' r f r {, Aeration �y fic _. I ! ! Boiler Blower Remote Condenser Aft I Field Tubes •_ _ Testing Is Am Steam Piping Guided Wavelength Testing wmu[[u[ur Centrifuge Hot Water Flush Exchanger Figure 40:Inspection of Steam Process High Value Assets and Support Equipment Informed Capital Projects Phasing 86 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 322 of 461 Figure 41 shows a remote field testing used to approximate wall thickness loss on 1,200 tubes. The results showed no detectable thickness loss and provided insight into RUL. The picture on the right shows Ultrasonic Thickness Testing for measurements on pipe elbows. Staff compared data from past efforts. The Guided wavelength testing on 100 ft (30.5 m) straight lengths of piping identifies corrosion and cracking by placing a meter on a 1 ft (0.3 m) section. Testing Method-Remote Field Testing s Testing Method-Guided Wave Inspections 46 loss* twow•.r. be*•••..' �.. Date of •• i • ' — l Figure 41:Central San Uses Advanced,Non-Invasive Technologies to Perform Condition Assessments and Minimize Operational Disruption 87 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 323 of 461 Page 90 of 195 During the implementation of Cityworks, an "Asset Condition Report" was developed using Crystal Reports, which is accessible through Cityworks.The report prompts minimum and maximum Condition Scores and then queries the Treatment Plant Entity Group in the TPFleet geodatabase to display assets matching the selected criteria. The report shows additional information such as reactive work order count and costs in the last year, the ratio of that cost to the asset cost, the install date, service life, COF, Risk, etc. The report allows Plant Maintenance to monitor the asset's health, issue corrective work orders for observations that can be addressed through maintenance activities or recommend replacement to the Engineering and Technical Services Department through a future capital project. Additionally, Engineering staff can run the report to compare against projects included in the CIB or CIP and adjust the scope accordingly. ' Central Contra Costa Asset Condition Report _ Sanitary District Minimum Condition Scare:7 Maximum Condition Scare 10 :6RM 41'C n�equenae Posibir4y Business Asset lD Legaty ID RM We Count Rh1 WO Cost A,—Gost Cas[lAsse, —di— Ccrtll,ion 1— stall 1— Ser,ite rife Fa of Faiure Risk Process Sash Frotess Srs,em Entity Type 1271 45741 Air Comp,Recytletl Serge lank 1 5422.6. 515,C00 OS 10 20`'i 199'_-0711 15 5 10 30 10 Clearv,el Reb:VVJSYs Ai-C�rnpre,cr 9102 74143 Air Comp,AUK Boiler l 1 $000 515,C00 0% 10 20'�-BS-0S 1995-06=Ji 15 6 10 80 7 Stea mGene Al,B0dler Ai�CmFresscr 5794 96008 AirComp HHW Oil Pump 1 $0.00 $6,000 0% 9 20'=-08-08 1997-07-01 15 1 9 9 8 Equipl,,n HHW A,Cmp,es, 1954 525273atteries-SWGR 52 1 S2,39224 $1,000 3% 10 20'�-BS-0S 193�_11-01 15 6 10 80 7 Pv Dist Sub52 B--erf 5613 313033atteries-SWGR 81 1 5469.92 $1,Co0 0% 10 20' 08-08 1994-01-01 15 9 10 90 i P,u)", SUC81 3a,ry 758 550083atteries-SWGR 33 2 5704.8E $1,Co0 IS 10 20' 08-08 20'=-08-18 15 7 10 70 i P, 1)", SUC55 3a,ry 2940 710953ottom 3RG F—lCen,er-Shaft 1 $00- 55,00- 0% 9 20'�-BS-0S 193�_11-01 30 4 9 36 6 Irtln Fsrra-es B—iig Figure 42:Sample Asset Condition Report ............................................................................................................................................................................................................................................................................................................................................... I. VISUAL CONDITION ASSESSMENT PROGRAM A visual condition assessment approach (Level 2) and inspection forms were developed with the CWMP (Carollo Engineers, 2017). These inspection forms were subsequently standardized and configured as Inspection Templates in Cityworks. Inspection Templates are configured by asset class, and each observation has a drop-down list of answers that are used to calculate a Condition Score ranging from 1 (good) to 100 (poor). Future enhancements include: • Replacing similar assessments is part of routing PM work orders with these Inspection Templates to improve data collection consistency and monitor the asset's health more frequently. • As a baseline and where feasible, all assets shall receive a visual condition assessment starting at 30% of service life consumed. ............................................................................................................................................................................................................................................................................................................................................... II. BURIED PIPELINES An on-going program was established to inspect the outfall pipe and make necessary repairs every 5 to 10 years, depending on the results of the last assessment. This condition assessment requires complicated shutdowns and temporary bypass pumping and piping. Condition assessment of buried pipelines (other than the outfall) on the treatment plant must be initiated. Staff will need to inspect the condition of several large-diameter, critical pipelines on the treatment plant site, such as primary effluent, mixed liquor, secondary effluent, final effluent pipelines, and wet weather bypass pipelines. These inspections will require complicated shutdowns and temporary bypass pumping and piping. After analyzing the data, the project team will recommend an appropriate re-inspection interval for buried pipelines at the treatment plant. 88 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 324 of 461 Page 91 of 195 C. RECYCLED WATER The condition assessment program for the recycled water treatment facilities will follow the program for the treatment plant since it has similar assets. In 2021, water meter inspections were performed as part of a larger project for pre-design improvements on the recycled water distribution system. The consultant was tasked to inspect each recycled water meter to perform condition assessments, site photos, and collect meter data such as model, serial number, and size. The end goal was to produce a list of out-of-date meters that need to be replaced so that Central San can implement a more efficient meter reading solution. The site photos can be tagged to the asset and viewed in GeoPortal. At the customer sites, CSOD staff conduct scheduled inspections and testing as follows: • Quarterly Site Inspections performed by the site supervisor for each customer site. • Quarterly Inspections for Clearwell. • Annual Site Inspection: A recurring work order is scheduled through Cityworks and includes an inspection for each customer site. The inspections are sent to the Planning Group for inclusion in the annual report required by the State General Order. • Four-Year Cross Connection Testing: Required and only for dual-plumbed facilities. Testing is scheduled by the Planning section. More information on the Cross-Connection Control Program can be found in Chapter 3 of the Recycled Water Program Manual (Central San, 2006). D. ASSET COST The asset cost is stored as an attribute to the vertical assets in the asset registry. The asset cost is for the asset only. It does not include engineering and administration, installation labor, related system modifications, etc., which are factored into the total project costa (i.e., replacement cost). The asset costs are intended to serve as a foundation for estimating the total project cost to be used in preliminary evaluations or studies. The asset costs will conform with a Class 4 or 5 cost estimate. The costs conform with the Association for the Advancement of Cost Engineering International's Recommended Practice No. 18R-97 (Cost Estimate Classification System -As Applied in Engineering, Procurement, and Construction for the Process Industries), as described in TM G-3, Basis of Cost Estimates, of the CWMP (Carollo Engineers, 2017). This memorandum also includes cost estimating guidelines for making life-cycle comparisons of project alternatives and establishing capital costs for the CIP and includes unit cost tables for linear assets. As projects are scoped, staff should refine their total project costs to Class 1 to 3 as appropriate. E. REPLACEMENT COST The replacement costs referred are a product of the asset cost and a multiplier based on TM G-3, Basis of Cost Estimates is as follows: 3 The asset costs or total project costs may not match the "book value" that accounting tracks due to different assumptions and methods.Accounting does not track individual asset values, but instead rolls up costs to a process level which is not supported by the existing financial system so Accounting uses a series of spreadsheets to calculate adjustments that are then entered into the financial system. 89 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 325 of 461 Page 92 of 195 • Linear assets x 2.195 • Vertical assets x 1.98 • Vehicles and equipment x 1.0 (i.e., no multiplier) ............................................................................................................................................................................................................................................................................................................................................... I. TOTAL PROJECT COST The most recent basis of cost estimates was completed in 2016 with the CWMP (Carollo Engineers, 2017) based on TM G-3, Basis of Cost Estimates. Staff will use the present worth life-cycle cost analysis to compare project alternatives. Capital costs include direct construction costs plus indirect costs required to implement the project. Indirect costs include engineering, legal services, environmental mitigation, Central San administrative costs, and project contingencies. Capital cost is synonymous with total project cost. Construction costs, such as the bid climate, materials or specialized labor availability, or site conditions, vary with changing conditions. To account for these uncertainties, an estimating contingency will be applied to direct costs. Annual O&M costs are the ongoing costs to run and maintain a facility, including power, labor, chemicals, and equipment replacement. O&M costs vary by treatment process and are a function of the flow or load treated. They will be used to compare the alternatives' life-cycle costs and establish annual operating budgets for the selected alternative. ............................................................................................................................................................................................................................................................................................................................................... II. RESIDUAL LIFE The residual life of an asset is the amount of time the asset can continue to reliably function with adequate maintenance in its current condition. The residual life is used to estimate the replacement date of the asset so that it can be incorporated into the CIP and CIB, which is then used to determine a funding strategy, rates, and fees. The residual life is typically calculated as the install date plus the asset's service life minus the install date year plus the current asset age. The expected service life can be estimated from published industry standards, the manufacturer, agency experience, or predictive forecasting. The actual service life of the asset will vary from the estimated service life depending on how it is operated and maintained, the environment where it is located, and the quality of its manufacture and installation. Predictive forecasting based on Central San asset data provides the most reliable service life estimate. This analysis was conducted for collection system gravity mains in the CWMP (Carollo Engineers, 2017). For all other assets, the Asset Management Section has reviewed industry standards, other agencies' experiences, and Central San's own to develop expected service life tables for all existing asset classes and subtypes. 5. EXECUTIVE LEVEL RISK INDICES For many years, leading utilities and other organizations have improved the ability to identify and address risks by creating centralized risk oversight Enterprise Risk Management (ERM). ERM is a holistic approach to identifying, defining, quantifying, and treating all the risks faced by an organization. Central San began implementing ERM in 2018 with the development of an operational risk register. This was followed by development of strategic risk inventory in 2019. Each strategic risk is assigned to a member of the senior management team. As a risk owner, they are responsible for developing and prioritizing mitigation plans for each assigned risk. The ERM Team meets twice annually to review and 90 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 326 of 461 Page 93 of 195 update all strategic risks and then reports notable changes to the Board. Of the 28 strategic risks, 8 are mitigated or controlled by the continuous and effective operation of assets across Central San's operation. More info on risk inventories can be found in Appendix 14. Enterprise Risk Index and in Appendix 15. Development of the Enterprise Risk Inventory. 91 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 327 of 461 Page 94 of 195 Decision-Making Criteria: Drivers and Life Cycle Management 1 . DRIVERS: AGING INFRASTRUCTURE, REGULATORY, SUSTAINABILITY AND OPTIMIZATION Assets can fail in several ways—capacity, age, efficiency, and physical mortality. Asset replacement drivers are conditions or circumstances that impact the ability of an existing asset to continue to meet its functional requirements. Projects included in the CIP address one or more of the four major drivers: • Aging Infrastructure: an individual piece of equipment reaches the end of its useful life. • Capacity: increases in process throughput may exceed its ability to meet current or future needs. • Regulatory: comply with regulations now and continue to adhere to them as they evolve or change in the future. • Sustainability & Optimization: optimize existing treatment processes, energy-efficient improvements, recovery of resources from solids processes, production of recycled water for refineries, and enhancements for service reliability during possible seismic and flooding events. Most project scopes include several project elements that address a range of drivers but make sense to group into a single capital project from a capital improvement perspective due to location, timing, or type of work. Aging Infrastructure — Capacity - In Evaluated O .� Progress Repair or replace some equipment • Expand the capacity and and structures to extend their redundancy (flows and useful life loads) of some no 00 equipment and facilities Sustainabilitylaptimilation • Optimize existing treatment processes Regulations • Energy-efficient improvements Comply with regulations and • Resource recovery O 01" continue to adhere to them • Recycled water as they evolve or become • Seismic and flooding event stricter in the future improvements Figure 43:Key Drivers for Long-Term Planning 2. CAPITAL IMPROVEMENT PROGRAM Annually, Central San updates its Ten-Year CIP and adopts a CIB. The CIP identifies and prioritizes capital projects needed to accomplish Central San's Strategic Plan and provides the basis for project scheduling, staffing, and long-range financial planning. The CIP also serves as the framework for rate- 92 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 328 of 461 Page 95 of 195 setting decisions based on planned expenditures. Projects are included in the CIP based on one or more of four drivers. Once a project is determined to be necessary, based on operational or maintenance needs or existing asset conditions, the project is listed in the CIB. The CIB includes expenditures for the planning, design, and construction of capital projects and is categorized into four programs: Collection System; Treatment Plant; General Improvements; and Recycled Water. The CIB is a detailed presentation of the project cost estimates proposed for the first year of the 10-year CIP. Central San funds an extensive Capital Improvement Program designed to preserve, maintain, and enhance assets, meet regulatory requirements, accommodate the community's needs, and protect public health and the environment. Capital improvements are construction or renovation activities that add value to fixed assets (pipelines, buildings, facilities, and equipment) or significantly extend their useful life. A. PROJECT PRIORITIZATION Each project is evaluated using a prioritization scoring system that includes input from stakeholders within the Operations and Engineering Departments. The prioritization scoring system uses existing Central San strategies, including guidelines developed by the Water Environment & Reuse Foundation. It also includes prioritization procedures like other NACWA member agencies. Each project is assigned a priority ranking. The criteria used to develop the scores consider the "triple bottom line plus" or social, environmental, financial, and technical benefits of the project and their applicability to Central San's Vision, Mission, and Values. New projects that were identified as part of the CWMP were prioritized through discussions and workshops with Central San staff. The new projects were then scheduled into the CIP based on their relative priorities. B. TREATMENT PLANT Regulatory drivers and needs are discussed in the CWMP (Carollo Engineers, 2017) in TM TP-1 1, Existing and Future Regulations. The primary regulatory driver for liquid stream facilities is meeting existing regulatory requirements and refining a long-term plan for potential nutrient reduction requirements. Process intensification technologies are being installed in other treatment plants in the Bay Area which could eliminate the need to relocate contaminated soils to free up land for new nutrient reduction facilities. Future potential regulations also include treatment processes for contaminants of emerging concern removal, such as ozone disinfection; however, these requirements are not anticipated within the planning horizon. Regulatory needs for the solids stream include improvements to meet existing and future stringent air regulations. 93 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 329 of 461 Page 96 of 195 C. PROJECT CLASSIFICATION Projects are grouped into the following improvement categories: • Capacity: These include replacing existing assets or adding assets to convey and treat peak wet weather flow from the wet weather scenario and volume from the design events. Projects that address peak wet weather flow and volume deficiencies within the planning period are considered capacity-related and not an expansion of the system to connect future users. It is also possible that treatment capacity can be exceeded due to increased organic loading from population growth, industrial user addition, etc. • Rehabilitation and Replacement: These include improvements to existing assets to extend their useful lives, repair structural deficiencies, or improve performance to provide adequate peak hydraulic capacity. • Planned: These include asset improvements and projects to meet regulatory requirements that Central San has scheduled to be completed within the planning period. • Annual: These include estimated project costs that Central San identified as annual projects that maintain their assets throughout the planning period. 3. ASSET LIFECYCLE An asset lifecycle is a series of stages for managing an asset throughout its lifespan, from asset creation to asset end-of-life. This Iifecycle follows Uptime Element's Reliabilityweb Asset Management timeline. A crucial element in Central San's maintenance program is having O&M staff involved in all asset Iifecycle phases. It became especially important to have O&M staff support the Planning, Design, Construction, and Start-up phases of capital projects. This "design for maintenance" would allow for significantly improved ease of maintenance and reliability once the new system or equipment was installed. Planning Case Ana OO c ® o�i3 F P "* 'L� Capital Renew Design Projects � Asset L_ -.A Operate, LifecycleFrConstruct on* Maintain, Create/ Capital Operations & •• Projects pgrade Maintenance Asset Management Figure 44:Asset Lifecycle 94 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 330 of 461 Page 97 of 195 A. BUSINESS CASE EVALUATION The Planning and Development Services Division initiates the planning phase of project, providing data and risked-based analysis through modeling or asset condition assessments, and use these analyses and models to identify assets for improvement projects. A business case evaluation (BCE) aims to conduct a structured analysis to recommend a course of action that best resolves a specific problem or issue, meets a defined challenge, or takes advantage of a unique opportunity. It is usually selected from among a set of feasible alternative strategies. Central San's Engineering Department developed a suitable CIP validation framework and methodology. BCEs consider all aspects of asset ownership, including service levels, capital costs, and O&M costs, and incorporate non-cost factors. BCEs are a rigorous method for evaluating alternatives applied at most levels of alternative analysis: from decisions between major project alternatives to selecting the types and numbers of assets used in a preferred alternative. In most cases, a BCE considers all aspects of asset ownership, including service levels, capital costs, and O&M costs. It also incorporates non-cost factors to ensure the most appropriate alternatives are selected and that the selection process is well documented. For major improvement projects, performing a BCE is a multi-step process that maximizes objective over the subjective assessment of project alternatives. This process starts with evaluating specific project needs and drivers and outlining the essential functional requirements for the proposed solution. Central San then conducts an in-depth evaluation of existing asset condition assessments and determines remaining life, evaluating whether to keep and maintain the existing assets; in- kind or replace them with new technology. After the evaluation period, Central San conducts a pilot test of the preferred technology when appropriate to validate design criteria and risk. It then evaluates current and possible future regulatory drivers that would impact the project and the identified solutions. Following the initial evaluation, Central San performs a preliminary investigation, such as geotechnical land survey, permit or environmental research, as needed. After that, the design alternatives and capital expenditure for each alternative is prepared. The Treatment Plant O&M for each alternative is evaluated to determine the pros and cons, and a 30- year net present value life cycle cost analysis is prepared. The conclusion of the BCE must justify a recommended alternative, which demonstrates how the recommended solution provides the best overall value for Central San ratepayers and the community when considering the level of service, economic, risk, and intangible factors. The project team then presents the recommended alternative to the Executive Team. After incorporating the recommendations of the Executive Team, the project team prepares a Board position paper and presents the recommended alternative to the appropriate Board committee. After incorporating the recommendations of the Board Committee, the project team refines the position paper and presents the recommended alternative to the full Board for approval. Appendix 11. Business Case Evaluation shows the process flow diagram for the BCE procedure employed at Central San. B. PLANNING/DESIGN The Capital Projects Division is under the Engineering and Technical Services Department. The Capital Projects Division engages with in-house staff to design linear assets for the Collection System Program. For Treatment Plant projects, the Capital Projects Division engages with design consultants to design vertical assets and systems. Project design phases consist of 10% conceptual, 30% preliminary, 50%detailed, 90%detailed final, and 100%final deliverables. For Treatment Plant projects, Maintenance staff selects the right equipment, like motor control centers 95 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 331 of 461 Page 98 of 195 (MCC) or standardizing equipment when feasible, and ensures maintenance needs such as access, lift points, and housekeeping elements are not overlooked. C. CONSTRUCTION: CREATE/ACQUIRE This asset lifecycle phase consists of constructing new assets and rehabilitating existing ones such as pipelines, general building improvements, or replacing aging infrastructure. The Capital Projects Division engage with consultants and contractors to create, acquire, and construct new assets in their capital and improvement projects. Plant Maintenance reviews the submittals and shutdown requests, provides job walks, and performs QA/QC of contractor construction work. D. ASSET HANDOVER At Central San, asset handover is critical to an asset's lifecycle. The assets created and acquired during projects require coordination among all Central San divisions to ensure the continuity of operations and maintenance of new assets. A comprehensive interdepartmental asset handoff protocol is a major component of plant maintenance involvement in capital projects. During the Planning, Design, Construction, and Project Closeout phases of acquiring and disposing of capital assets, asset handover requires working across departments and various programs. Staff across divisions work with Asset Management to catalog and assign necessary data attributes to new assets and document which assets will be disposed. The Asset Management team then incorporates this information into the asset registry, which is accessible through GeoPortal and Central San's CMMS. The Asset Handover process ensures that assets and their associated information, like complete PM schedules and documentation, are transferred from Capital Projects to O&M teams once installation or construction work is complete. This protocol ensures that assets, including complete PM schedules and documentation, were transferred correctly from Capital Projects to the relevant team upon completion of installation work. This asset handover protocol worked well within the organization, but it became clear that contracted work should follow the same process. The improved asset handover protocol would also include specifications to the required standards expected from the contractor and delivered to Central San as part of the asset handover workflow. The Asset Handover workflows for Collection System Pre-Construction and Post-Construction for Capital Projects and Developer Jobs are depicted in Appendix 8. Collection System Asset Handover Workflows. The Asset Handover workflows for Treatment Plant assets are shown in Appendix 9. Treatment Plant Asset Handover Workflow. E. OPERATE, MAINTAIN, MODIFY, AND UPGRADE After asset installation and acquisition, assets are operated to achieve their functional specifications. Operators and Maintenance staff optimize the value of an asset, maintain it to extend its lifecycle, and ensure risks are controlled. As assets spend 80% of their lifecycle during this phase, regular inspections, condition assessments, and planned maintenance activities help optimize the assets' lifecycle operational life to deliver reliable, high-quality services in a sustainable manner. F. REMOVE AND/OR REPLACE This phase comes when Central San needs to demolish and dispose of assets through end-of-life projects, capital improvement projects, or abandonment when no longer in use or needed. The 96 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 332 of 461 Page 99 of 195 asset is safely taken out of service following all environmental, regulatory, and health requirements and controls during decommissioning to reduce the impact of hazardous and non-hazardous materials on staff, the environment, and community. Procurement manages the controls. The asset removals are included in the replacement project contractor's scope of work. The asset status is captured in the asset registry to ensure asset and related maintenance information and activities are intact. Inspection and maintenance activities linked to the asset remain in the CMMS indefinitely, and the asset is kept in the asset registry with historic, retired, or disposed status. When an asset is taken out of service for maintenance, the Asset Management workgroup, working with Plant Maintenance Planners, updates the asset registry to out-of-service or abandoned, depending on whether it's Treatment Plant asset or Collection System asset. 4. REMAINING USEFUL LIFE The RUL of an asset is the most limiting useful life associated with the four replacement drivers. For example, assume the following for an asset: Table 13: Remaining Useful Life Remaining - - Year Aging Infrastructure RUL: >20 years Capacity RUL: >20 years Regulations RUL: >20 years Sustainability/Optimization 10-20 RUL: years For this example, the most limiting RUL is the one associated with the Sustainability/Optimization driver. This asset may need replacement within 10 to 20 years, even though the other drivers all indicate a RUL greater than 20 years. Thus, RUL is limited by the lowest RUL of the four drivers. Each of the above drivers have multiple considerations or sub-drivers when developing the estimated useful life. For example, the Sustainability/Optimization driver has four subcategories (safety, operating ease, maintenance/redundancy/reliability, and long-term flexibility). The average RUL values across all four subcategories was used to quantify the Sustainability/Optimization RUL. Capacity may have treatment capacity and hydraulic capacity considerations. Defining typical lifespans of equipment is instrumental in deriving the estimated RUL figures. The evaluation lifespans provide uniform, generally middle-of-the-road benchmarks against which to score equipment. The basis RUL estimates is explained in condition assessment and comprehensive evaluation reports to assist future evaluations. Failures can and will occur in advance of nominal lifespans, and it is also expected that equipment outlives the ages listed above, which is the case in the Treatment Plant. The objective is to define reasonable expectations of how long equipment should be relied upon. Table 14:Nominal Lifespans of Equipment Equipment Nominal Lifespan Low-voltage Secondary:5600-volts Dry Transformers 30 Low-voltage 480/277-Volt Distribution Equipment 30 97 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 333 of 461 Page 100 of 195 Low-voltage 480/277-Volt Motor Control Centers 20 Low-voltage 600-Volt Conductors 30 Medium-Voltage Oil-Filled Transformers 35 Medium-Voltage Distribution Equipment 40 Medium-Voltage(5kV and 15kV Conductors 35 Low-Voltage Variable Frequency Drives 15 Condition assessments were performed to evaluate the RUL of the equipment and identify improvement areas. The condition assessment results inform the RUL organized into four (4) groups based on the replacement time frame, as shown in Figure 45. Assets highlighted in red could be replaced "immediately" in the next five years, those -Oqq highlighted in orange could be replaced in the "short-term" in 5 to 10 years, and those in blue could be replaced in the "long-term" in 10 to 15 years, and those in green can be replaced after Immediate Near-Term 20years. (C5 yrs) (5-10 yrs) Options that could extend the RUL (without equipment replacement) but generally perpetuate aging equipment that meets obsolete standards and falls short of today's Future Long-Term standards. (>20 yrs) (10-20 yrs) Useful life and RUL (from HDR) - based on condition assessment and non-destructive testing such as guided wave testing, which helps determine longer life and add weightage percentage. Figure 45:Anticipated Remaining Years of Useful Life 5. MAINTENANCE HISTORY An asset's maintenance history supports the condition assessment inspections. Cityworks was used to pull equipment history for all steam system assets. As an example, the Steam Project conditions assessments categorized work orders not associated with predictive or preventative maintenance, such as reactive and corrective, as "unscheduled." The failure incidents were plotted over time to develop a trend so that their relative position on an equipment failure rate curve ("bathtub" curve) can be determined, as shown in Figure 46. If the incident's curve demonstrated a downward trend, the equipment resides on the left side of the curve and has a remaining useful > 50%. If the incidents increased over time with an upward trend, then the equipment resided on the right side of the "bathtub" curve and has <50%of RUL left. 98 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 334 of 461 Page 101 of 195 Decreasing Constant Increasing Failure I Failure I Failure Rate I Rate I Rate � I i I l I I 1 ` M *Early ! Observed Failure 1 �.�• 1:9 *"Infant Rate • a L y Mortality I I •• Wear Out 3 Failure I ,+ • a� Failures Constant (Random) *.• U. : I Failures41 I +•• �. ir``*.■.*"y �■■rr■■�■■ra■ram■r■■ri■■■■■�■■a■■�■wig■" Time Figure 46:Equipment Failure Rate"Bathtub"Curve 6. DATA MANAGEMENT AND UTILIZATION FOR DECISION MAKING A. DASHBOARDS AND ANALYTICS Dynamic dashboards for CSO and Plant Maintenance showing trends and KPIs are a valuable resource for decision-making. Central San replaced a manual paper process of creating monthly reports with a single dashboard with multiple tabs. Previously, staff would compile an End of Month Report for CSOD Field Operations every month. This task was time-consuming and required gathering information from multiple sources. With dashboards, the data is pulled directly from the relational databases used by the CMMS and ITpipes sewer inspection system. The dashboard shows data in near real-time and is refreshed every 15 minutes. Staff use the dashboard to plan work schedules, evaluate work activities, analyze historical information to decide corrective actions, and make decisions. 99 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 335 of 461 Page 102 of 195 SCNEWLED ClwN"RDOT/iff CampINWM H,(ITEM?, SCHEDULED tlpning WORKORDERSCempletW Memhly-Ct88FNTNFaRpTEM61 SCNECAEDtlaaning FODTAft fnn,plalW Memhy-<]tRRFNT 'ITFM6) .. sessas •.. x ;zn i z s � s A sas� t..pMt sr Pw An s.c w. -,.i.amt s1, nw xw %v rvo. .Invro �Hyrnt .w00irvc �xwao .torat .aocdsn NSC,IEWLEO Clee n9 yGEConp'- 'Mnully(1IEN]i �UNSCNEDULED Cleaning WORK CODERS CompletedM 1y-nivarvr eaIREM 61 UMCHEDULEDCIeaning FOCTADECompi,WMemhiy.CURRENTYEARIII£M6) 01 Figure 47:CSO Cleaning Dashboard Showing Scheduled and Unscheduled Activities .Dashboards lime Period Oven WO arSGJ of Period Oven Footage ar SteH of Period [°mvie dW0[°Da,e Thia Completed C.n Open F°Wage Open Hadi°g LYO 6adbg WO Peiod CnniPle,ed Curent Month 6 11 142,283 207 51 ,695 7 a RODDING Curre nt Week 552 130,328 148 404 90,588 WaRordep psnsa Moje�dFi„isM1:dAOI'102,,t,:55:00 PM scam 4ary HO1D 39,740 -Cted Pln�en.bnangzo,uoo:poAM uH�pfNo,CirvworRs work order 1tme PCAod Open WO mSnrt WPenod Ope Fantage ar5mrt of Pennd CampktedYw to DPre Thu Campitted Foomgeisiavenod r rren r wn Cunxm°pen somage OpeP Raddogvy0 Period ComplMeJ Current Month 660 150,849 306 70,893 3 0 HYDRO Current 354 79,956 WoM1OiJan d9838fi Week Projected TinieM1:M30R021,12,Oo:00AM 502 110,794 48 30,838aA551GNEb ° Woht Orders 56 rojecred Finish:10/T I/]011,13:00.00 ere fnrNErvw Figure 48:CSO Preventative Maintenance Scheduled Weekly Dashboard with Indicators and Backlog Linked to Computerized Maintenance Management System The CSO Maintenance Planner uses the PM Scheduled Weekly dashboard on Mondays with CSO Supervisors to review the completed work. Work orders are assigned on the first of the month. 100 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 336 of 461 Page 103 of 195 CSO Dashboards G NFao,age Completed byMwnA(REM 9] Month-To-Date of 51378 J 'II Feet Completed(6y CSO) I I I IIIIIIIIII61�IIIIIIIII I I IIdJI III I I III I III I III-IIIII I�IIIIIII�I-I. ry nzpecnom completed 6y MomA Month-To-Date 20 Inspections Completed(by C50) I I IIIeI II III II I�IIII,IIII I�IIIIIIllllll, I 1 IIII I III I III l Ili��I_���IJ��� 1 Figure 49:CSO Closed-Circuit Television Dashboard with Indicators and Graphs for CCTV Inspections by Staff or Contractors Staff developed dashboards to provide work order backlog metrics, which is critical for planning future maintenance work and guiding work allocation. It also provides insights for resource allocation and workforce management, as well as easily accessible and intuitive metrics to track the performance of various shops. For a use case, the backlog dashboard staff may compare annual work orders or quarterly statistics to see the effects of pandemics or endemics. Maintenance Planners use the dashboard to review the various shops and utilize the work order hyperlinks embedded in lists to address the backlog work orders. 7ork Orders Closed by shop 2i'-12 Total Inwunnent .. • 52 WOs Closed WOs Closed Machine .. .. 40 Wos Closed WOs Closed Mechanical Shop . . 1 WOs Closed WOs Closed Figure 50:Treatment Plant Maintenance Dashboard of Work Orders Closed by Shops 101 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 337 of 461 ^mP . TPBaeklog C •rnae Maintenance Backlog Dashboard NRercl..rbbroahRarrsa:',w+rsamr,sArm2+ - r m T 16ad1pR York O+derz(O:nentY®rl I� am• • Total Backlog Work Orders Total Backlog Work Orders Tota I 15 8 roul paddpawmkordkraRksaYean Actual Hrs>Estimated Hrs No Estimated Hours • � n>dwem.mr ud �P Backlog Work Orders Taal&iddeg WekOmera Wham Ac+ual Ttxl Badiep Wmk ONemWhNu I �"'�-..�-...r.✓x!-�`-""`"-�` r.,bnrH..>Fn,rr.,r¢AlahorHrs Fnm,tnAlas+nrHn:,rs w:dvr.r..d.y tm + ++ffi AreMn P us9° x pnc� I s TI Badiog Hasn(Cu,rentTearj 1 z Total X— � irP���p- 1 s^ti/ �✓ fJ ' Y+ o aPra M.m �.nl sro=, N,.=1 • • k 1 ,915.5 Backlog Hours • • riz ofY day tY ar,to roi ra /w9 to sv'to auto norm p: —' � ° oacP ° uc. TPBacklog •r^- Maintenance Backlog Dashboard nnarcnaru Wpam Range: �;;a: -_.�__� r mdilpmw rnmarwa sl.n e.a*feg rxodr r]i.m padOPa Werh GmMm lkAryl wnN nrtMrF shnp'.MEcxANlCpr sM1xPMneNwS Total Backlog Work Orders Tolal Backlog Work Orders peasprron PM NR Syaum Nut%COWer Oe-iPoon 1 A.raLi Meaxrramamlv.„ Seatt.z INITIATED 5 s.INRIATCp Pre acred NBr31ITOT1,B;p0�pp 0.M gaRetlF � R/]vLDll.Klp:3r+h Prrl 15 8 Shop',MCCIIANICAl 51nP�Mn['NINS �piNirinrEn rx:®AaR®1exrsmar o. muya,nnum,p.;r actual Hm>Estimated Hrs No Estimated Hours Prcleaed Finizhl 1!]NM21.5:00:W PM S+rtus:IN PROGR[SS " - � nga<tanF�a�en,mn]xoal,lauonax shnp.MEenAMlct4 ... .... ..- tiel Veale,tlar unl YeN.m.y warp Omer:3r5eds P oMaanM c.r A+am 2wr. relalx., t Y,.:la.,whirr. krt.l r,y knrda.a wen i ProiectM rn�zh.9rl Tl'1p21,Y:00:00 PM peacr:peon:T ouGeaM1oevRep�r IN PROGRESS ShoP',MECIIANICILL ,. Ipr29rfaat,—aM In wprY Omer:45]Re6 -. .�� p°anlPtion:PM SwmNart%C 1Ywr Beetlog WnY Omm evdlop Work OrJrz r• + Eo Pbiu:_. . r eormzl.s:mog,w shay:EYE.s Hoop:as - Ier LYr.ork.vJarh Ortl.r Wort Ortler:3}B16 ea6 vnak Ordasro985°s r yrop-MEgVaa('0.r —ripti°n:PM furnace Tu+,xwntl Operrtiona Oaacripdon l`M l9anz ManM1°le.li3 aM GradrglYear °rlr Aden 0591.s3 0. er+m ❑ew'�pk°rr:PM L'sn9w Yi-11.z lYear 'InmaTEo Psogeaee Nnl n•:lve]2g91,s:o3:opAM :° Saws:VNm0.T£D Ppip{ta:d FIPM:17/4Yi931.d:1o;p0 AM Wnixzed NKM.B/aW0021,5:00:00 PM rJrrl�i mot:. .. SLnR 90G SHOP .• -- ------ `--sue--- SIaeR ELfCSNOP Nmrk Order:531711 ° ffiopi MEo,ANIEx VMM1 ONen µfAt¢ peav�prNon�capwl Prgw Support � 'I� WprY Omer:4619]6 peoopkon TrsrMwromrRepair Plarrc Seeerlghtirq 5 us: r e . .�P Daslpkon:PM IGe smr a lber—.—I Shop SYm.cN NOI➢ Prgxr.d RnirM1:19r0rT0>I,+6.90T0 r.M PmpabdFlmh:9RAl2021.6:Og:W R10 - ' Pmpxmd Fin sn elllrn0st,5. M rr--- -- .• shop:ELEc sN 51zPVl ME.x;.vit,�: Wb:h Omar:a]31a1 work O Fenl Nemrr lnna 1 Ye¢r pewlpp°n:Tonhrea�aovRepai, 5,a,us NTIATY SNrrn:tNRIATfp Hnra.nirl Fi�ris�.Isranf>51 aULLLp PM • Page 105 of 195 C. SMART MOTOR CONTROL CENTERS Smart MCC and Switchgear connect to Central San's SCADA network to capture live data and help staff make informed decisions. Advanced diagnostics improve predictive maintenance and ensures improved safety, uptime, reliability, and equipment longevity. 103 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 339 of 461 Page 106 of 195 APPROACHESASSET MANAGEMENT 1 . OPERATIONS STRATEGIES A. COLLECTION SYSTEM - PUMPING STATIONS The Pumping Stations group operates pumping stations within the Collection System Operations Division of the Operations Department. The staff rotate assignments and ensure that all stations are visited each workday, complete rounds-sheets, and conduct maintenance as needed. During off-hours, an operator is "on-call" to handle any problems. The pumping stations use Wonderware software, which the Operators can access remotely or at the pumping station. All the pumping stations have either a level transducer or a bubbler system to monitor the level of the wet well. These are linked to the SCADA system, which will send alarms directly to staff. B. TREATMENT PLANT The Control Systems group supports the Dynac and Wonderware software. Operations manuals are posted on the Plant Information Management System, an intranet site and added to Laserfiche. The Plant Operations Superintendent manages the day-to-day operations of the treatment plant. Four or more Operators are always on-shift, with a minimum of a Shift Supervisor in the control room and three Operators assigned to specific areas of the treatment plant. The Shift Supervisor is responsible for monitoring the treatment plant control system. The Plant Operations Training Coordinator manages the Plant Information Management System. The Control Group manages SCADA. C. RECYCLED WATER DISTRIBUTION The recycled water distribution system is managed by the Associate Engineer in the Planning and Development Services Division. The Associate Engineer schedules maintenance and repairs, works directly with the recycled water customers, and manages the billing. 2. MAINTENANCE STRATEGIES Maintenance management involves two fundamental practices- unplanned and planned. Asset Management best practices for decision making uses the Pareto analysis, where 80%of benefits can be achieved by doing 20% of the work or conversely, 80% of problems can be traced to 20% of the causes. Thus, in maintenance, if 80% of the cause of breakdowns are due to 20%of possible causes, spending a majority of resources on planned maintenance to identify the 20%causes will help to reduce breakdowns. As such, it is essential to manage and track unique maintenance types. Central San uses the following industry-standard maintenance activity category: 104 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 340 of 461 Page 107 of 195 • Reactive Maintenance (RM): Emergency-based work on equipment ■ or asset. RM occurs when someone Unplanned . Planned ■ interacts with or recognizes a problem ■ with an asset or infrastructure system and is almost always unplanned. This ""' ""■ ""' ----- may also have been called "unscheduled work" or "schedule RM PM R D busters". Reactive Preventive Predictive • Preventative Maintenance (PM): PM Maintenance Maintenance Maintenanoe action is conducted to keep equipment or assets working and extend the life of the equipment or asset. PM may include tests, measurements, Corrective f•.1�int�n�nr2 adjustments, cleaning, and parts replacement, performed specifically to Figure 53:Work Order Types prevent faults. • Predictive Maintenance (PdM): PdM is used to predict failures before they occur to keep unplanned maintenance to a minimum because unplanned maintenance costs 8 to 10 times more than planned maintenance. Equipment with rotating elements is a natural fit for vibration analysis, electrical equipment for thermography, and so forth. In some cases, the devices monitoring the asset may be connected to a building automation system, a distributed control system, or a Programmable Logic Controllers system. PdM is more advanced techniques and technologies that diagnose the current state of equipment and aid in predicting when equipment failure might occur. PdM includes trend analysis, pattern recognition, and other correlation analysis. Ideally, PdM allows the maintenance frequency to be as low as possible to prevent unplanned reactive maintenance without incurring costs associated with excessive PM. • Condition-Based Maintenance (CbM): CbM is a non-intrusive task performed to check equipment health by testing, monitoring or trending parameters known to be indicators of asset health. CbM can be online continuously or based on periodic inspection. CbM optimizes preventative and corrective actions by avoiding traditional run time or calendar-based maintenance tasks. At Central San, staff uses CbM and PdM interchangeably in the CMMS. • Corrective Maintenance (CM): CM may be a child activity stemming from a RM, PM, or PdM. It is any maintenance activity required to correct a failure that could occur or is in the process of occurring, to ensure the asset or equipment is restored to a satisfactory working condition. Often, CM includes equipment and asset repairs, but it may also include replacement. CM can be subdivided into immediate or deferred CM (the latter is delayed based on a given maintenance or capital improvement rule). The Plant Maintenance Division performs CM on assets. The work is usually identified by POD staff or when Plant Maintenance staff performs PM or PdM on the asset. • Asset Condition Management (ACM): ACM is the overall approach to monitoring, diagnosing, and identifying corrective actions. This can be done by continuous or periodic measurement and interpretation of data to indicate the asset condition (health) and determine its maintenance requirements. ACM includes accumulating and correlating 105 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 341 of 461 Page 108 of 195 asset condition information and observations over the asset's life. ACM includes PdM, CbM, and other maintenance technologies and techniques. • Asset Condition Information (ACI): ACI includes data and observations on the conditions of an asset. More than just the current state, it is data and information accumulated over the life of an asset. It helps support day-to-day and long-term decisions about asset conditions and future conditions. Additionally, troubleshoot/repair work order templates define failure codes for reactive and corrective work orders. Failure codes capture information about equipment, asset breakdowns, or failures and enable staff to document failure modes, failure causes, and perform reporting. These codes help provide the data necessary to build reports for statistical analysis and is extremely valuable for decision-making regarding stocking spare parts, standardization or sole sourcing of equipment, and prioritization of work. They also allow staff to gain insight into chronic issues, often not easily found, affecting a piece of equipment and help guide staff to a fix. The CMMS generates preventive and predictive maintenance work orders based on a frequency configured by the stewards (Maintenance Planners and Fleet Shop Supervisor). The work order history is stored against the asset in the CMMS for easy retrieval. Predictive work orders on specific equipment are scheduled through the CMMS for Plant Maintenance Division staff, including oil analysis, thermography, and vibration analysis. Corrective maintenance work orders are created as needed by staff. Work order history is stored against the asset in the CMMS for easy retrieval. If the remedial work originated from another work order, it is created as a child work order to retain the linkage between the work activities. Reactive work orders are troubleshoot/repairs in response to equipment failures or assets not functioning correctly. Staff uses Cityworks to manage and schedule all PM tasks to minimize reactive work. In 2021, staff completed over 5,000 work orders at the treatment plant; only 2% of those were reactive work orders. All maintenance activities are captured in Cityworks. Templates for routine work activities include instructions or an attached SOP document for quick access by the staff performing the work. The overall workflow model for the system is shown in Figure 54: Cityworks Workflow Model. The stakeholders also documented and agreed upon more detailed workflows at each workflow phase. A Cityworks Custom Business Process Manual (Lawson, Cityworks Custom Business Process Manual, 2017) was developed in 2017 to document SOPS for using Cityworks and include the workflows described above. The document is in Laserfiche. 3. MAINTENANCE PROCEDURES A. TREATMENT PLANT, PUMPING STATIONS, AND GENERAL FACILITIES Maintenance procedures for the Treatment Plant are attached to work order templates in Cityworks. B. RECYCLED WATER Maintenance procedures for valve exercising, system testing, and other tasks are managed by CSOD staff. CSOD staff working on the distribution system carry the recycled water distribution system O&M manual system maps in their vehicles. Additional maintenance information can be 106 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 342 of 461 Page 109 of 195 found in the Recycled Water Program Manual (Central San, 2006). If there is a repair that CSOD cannot perform, staff will call Contra Costa Water District to perform the repairs. Procedures for leak adjustments and hydrant truck fill program are stored in Laserfiche under the SOP folders and are grouped under Planning and Development Services, Planning and Applied Research, and Recycled Water Planning. 107 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 343 of 461 r Central Cordra Costa T-r4 To-Be Workflow Modeling Sanitaryustdo Enterprise Cityworks Implementation W O O L P E R T `^ul [^c! YY ....................................................................Notes I I .Primary Cityworks AMS users by Core Wok lowrocess P 1 1 re identified as follows: I I .initiation-CS Mai ntenance Sc heduler,(CSO) 1 1 I I Administratire Technirian,Maintenance Plainer,and/or supervisor Legend Preventative •Emergency-CS Maintenance Scheduler and[CSO] Asset Set-Up AdminiArativeTechnician,Supervisor,Maintenance Maintenance [ J pervisor and -- Planner a d or Lso sit �... ' A9odification Set-Up and Crre worldlovr ceo•ion Point •Screening-Supervisor,Maintenance Planner and/or ICSOF ; Modification supervisor I 1 1 •Plaming-Supervisor,Maintenance Planner and/or(CSO) I 1 1 Supervisor 5 Jredulin 5 ft Maintenance Planner anNor c�Pmrn I R-uperroor, (Not in E. � Refit II , •(CSO)Supervisor nw6s5<�I Process 1 1 •Assignment-Supervisor,Maintenance Planner,Field WI 1 WI t operations superintendent anNor(CSo)Supervisor I 1 1 •Performance-Maintenance Technician ntenan,Crew Leader, I I Maintenance Planner andlnr Maintenance Supervisor I I I •Q/u Review rr ComPletion-Supervisor,Maintenance I j j Pl.iner,Superintersdent,and/or(CSO)Supervisor I 1 1 •Asset Set-Up and Modification-CS Maintenance Scheduler Administrative Technician S( 1 and CSO J upemsor, P Apptiradon 1 o I I I Maintenance Planner and/or[C50]Srpervisor papyaye 1 1 Eiristirg lnrentory 1 •Preventative Maintenance Set-Up and Modification-CS Maintenance Scheduler and(CSO)Administrative I Noce'-T� Technician,Supervisor,Maintenance Planner andlor (CSO)Supervisor j •Vehicle-Equipment Inspections-➢river I Equipment 1 1 operator I I o I 1 I I Follow-Up 1 Folfalr-Up j Utfmrl Work I Work Work ' 1 Initiation ak Planning Scheduling Assignment Performance RA Review& Completion I I I '------'I-- I I CanCefled j Concefled PM Skippers I Work I I I i I i i I I I 1 I � I I I I I I I ' I I 1 __________Canfinrred XorrPrinrityriork____________________________ 1 -----------rrioriry Work•--------- Emergency Created Ry: gvyla Foutch Revision History: Date: August,7A15 Rev 1:QF-Update'.VOrkFlows Post WorlLsFiopGlobal Workflows Overview Figure 54:Cityworks Workflow Model 108 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 344 of 461 Page 111 of 195 4. COLLECTION SYSTEM A. SEWER SYSTEM MANAGEMENT PLAN Central San prepared a Sewer System Management Plan (SSMP) (Central San, 2022) following the requirements of the State Water Resources Control Board Order No. 2006-0003-DWQ, Statewide General Waste Discharge Requirements for Sanitary Sewer Systems. The order was updated and adopted on December 6, 2022 (2022-0103-DWQ) effective June 5, 2023. The SSMP is audited annually, updated every five years, and posted on Central San's website. The most current SSMP (Central San, 2022) details the O&M program for the collection system and includes the overflow emergency response plan, fats oils, and grease control program, design and construction standards, capacity management, etc.The goals of the SSMP are to: • Responsibly manage, operate, and maintain all parts of the wastewater collection system and adhere to the components of the SSMP • Provide adequate capacity to convey design peak flows • Minimize the frequency of sewer system overflows • Mitigate the impact of sewer system overflows • Develop annual strategic goals and measure progress toward those goals B. COLLECTION SYSTEM Maintenance procedures for the collection system are stored in Laserfiche under the SOP folders and are grouped as CCTV and CMMS, Field Operations, and Fleet Services. In addition, some work order templates have instructions and attached inspection templates. C. PREVENTATIVE MAINTENANCE Central San's operation and maintenance program is detailed in Element 6 of the SSMP. CSOD aims to clean all pipes 18" and smaller at least once every 84 months. The maintenance schedules are adjusted based on the pipes' history, results of CCTV inspection, proximity to environmentally sensitive areas, recent renovations or repairs, and other factors. CSOD conducts the cleaning and maintenance of sewer pipes with an approach collectively described as "quality cleaning." CSOD incorporates standard QA/QC measures to improve and maintain high cleaning effectiveness. Supervisors make frequent field visits to review cleaning crew practices and equipment operation. Supervisors complete weekly reports and provide feedback to cleaning crews. The interaction is similar to a coaching session rather than a performance audit. In addition, each month, randomly selected recently cleaned sewers are inspected by CCTV to check cleaning effectiveness. Sewer video inspections provide a proof- positive method to verify pipe cleaning results. The videos are viewed at bi-monthly Operations meetings and Supervisors share the results in a one-on-one session with crew leaders. CSOD staff also reviews CCTV videos to confirm that maintenance is scheduled at the optimum frequency. Different cleaning tools are evaluated and the most effective tools for various situations are identified. 109 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 345 of 461 Page 112 of 195 Further details about CSOD's maintenance strategies can be found in the SSMP, including pollution source control, root control, odor control, cleaning of force mains, cathodic protection, corrosion control, and how cleaning frequencies are adjusted. CSOD also has blanket service contracts for small, cured-in-place pipe and pipe-burst repairs. D. CORRECTIVE MAINTENANCE CSOD completes spot repairs in pipe segments and repairs/replacements. The typical length of these spot repairs is less than 10 feet. CSOD maintains a warehouse of material for repairs in Walnut Creek. Whenever possible, pumps and other equipment are standardized to reduce the complexity of working with different types of equipment. E. SCHEDULING - CLEANING OPTIMIZATION APPLICATION Central San has a proactive approach to preventive maintenance. All 18" in diameter or less lines are cleaned on a 1 to 84-month interval. For most of the 2000s, scheduling was done using an Excel spreadsheet, and CSO staff couldn't efficiently assess how to schedule pipe cleaning while also considering adjacent pipes cleaning schedules effectively. It is best practice to set adjacent pipes on coordinated schedules to reduce cleaning crews' "windshield time" or time spent driving between job sites. However, this was effectively not possible for most of CSO's history. That was before 2018 when Asset Management developed a web mapping application to provide a visual representation of cleaning based on location, existing frequencies, and type of cleaning. CSOD's Maintenance Planner uses this application to align cleaning schedules within specific geographic or neighborhood areas. This alignment increases footage to pipe cleaned per month by enabling crews to efficiently clean adjacent or nearby pipes in fewer visits to the same area and decrease driving time. Improved scheduling has the added benefit of reducing personnel overtime by enabling more cleaning during regular working hours and assisting CSO in meeting its Strategic Plan goal of completing more than 95% of scheduled cleanings on time. 5. TREATMENT PLANT, PUMPING STATIONS, AND GENERAL IMPROVEMENTS The Plant Maintenance Division performs routine maintenance on mechanical, electrical, and instrumentation assets based on fixed intervals, equipment run-time hours, or decreased performance (condition-based). An asset spends about 80%of its time under this operations and maintenance phase. An effective maintenance strategy is critical to the Central San's Asset Management Program's success. As a part of this improved maintenance strategy, plant maintenance was overhauled with industry-leading techniques in the following years, including automated maintenance reporting, oil analysis, equipment vibration analysis, and improved data management. In recent years, Central San has built upon its original goals, reflecting a renewed focus on improving its reliability and Asset Management Programs. These improvements included Asset Condition Monitoring (ACM) and Defect Elimination (DE) initiatives, Root Cause Analysis (RCA), and Planned Maintenance Optimization (PMO) initiatives and enhancing monitoring and reporting capabilities in CMMS and GeoPortal. Involving O&M staff in all phases of the asset lifecycle and using condition assessment has helped guide capital project decision making, enhanced monitoring and reporting capabilities with Central San's CMMS and GeoPortal, further involvement of O&M staff in all phases of the asset lifecycle, and use of condition assessment to guide capital project decisions. 110 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 346 of 461 Page 113 of 195 Figure 55 describes the reliability strategy for the Treatment Plant Maintenance and Pumping Station Program. It is part of a systematic approach to developing a comprehensive maintenance program based on asset criticality and consequence of failure. It can be achieved by a Reliability Centered Maintenance qualitative decision methodology that identifies the most effective preventative maintenance task for plant equipment and systems. Reliability Centered Maintenance with a CMMS establishes a repeatable program with documented processes and procedures. The Pareto diagram is the reactive maintenance on the Treatment Plant from August 2019-August 2020. 1 1 1 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 347 of 461 Page 114 of 195 Corrective Maintenance on Plant Systems August 2019- August 2020 $120,000.00 50 c ParetoChart 45 $100,000.00 40 O c � 35 0 $80,000.00 30 oi p W $60,000.00 25 L L U 20 cg o $40,000.00 15 cg U $20,000.00 10 J J J J � 5 ~ $0.00 0 O\5` r\Z�7 �`� �09 �De era\ Qt` as, ��c �it� C� ��� \a�� o�� �Q �? o oao� a�`o ono 0 20/80 Systems Bad Actors L A Better to Well Behaved System ECM • Function Driven • Data Driven Analysis by ECM Team • All Likely Failure Modes Analyzed • Existing PM/PDM Results and Historical CM's Analysis for Validation, Change or Addition of Failure Mitigating • Applicable and Most Cost-Effective Tasks Tasks Implemented Perform as per strategic plan; complete 5 RE per 2 year period • Perform as per strategic plan; complete RTI (TBD) 5 RE per 2 year period • Qualitative—O&M Subject Matter Expert Team Identifies Potential "Show Stoppers" Using 7 Risk Criteria • Performed on systems upon which ECM not yet done __J\ RCA SQE Defect Elimination (TBD) • Regulatory, Safety, Cost, Production, Labor/ • Safety, Quality, Efficiency(Internal Audit) • Repeat Failures from Known Unreliable Parts Time • Some Covered With Job Hazard Analysis and/or Practices • Goal Oriented Cause & Effect & Process • Performed on Large Jobs, New Jobs, and Routine • Raw Materials, Operations, Workmanship, Analysis by Team Audits Design, Collateral Damage Examined as Needed • Perform When Cause(s) Uncertain or high • Perform as needed • Perform as part of routine continuous criticality failure improvement Figure 55:Reliability Strategic Framework 112 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 348 of 461 Page 115 of 195 A. RELIABILITY CENTERED MAINTENANCE Reliability Centered Maintenance (RCM) is a systematic approach to developing a comprehensive maintenance program based on asset criticality and COF. This method utilizes qualitative decision methodology that identifies the most effective PM task for plant equipment and systems. Because of the analysis and software, it establishes a repeatable program with documented processes and procedures. Key points include: • Systemic approach to developing maintenance activities by assessing asset criticality • Reviews the effectiveness of existing and new maintenance tasks to prevent functional failure of critical assets • Preventing functional failure is achieved by understanding failure modes and focusing maintenance tasks on predicting and preventing failures at the earliest possible time (on set of equipment degradation with use of advanced diagnostics) before they cause significant negative impacts on service levels, compliance, public safety and environment, and employee safety. In addition: o By finding and correcting problems earlier, it costs less to perform this maintenance on the asset than it would to perform emergency maintenance. o Earliest detection and correction of problems will minimize costly collateral damage and provide longer asset life since emergency maintenance is often not able to restore the asset because its condition is beyond repair or collateral damage from the failure has limited the options for repair. o By focusing on understanding failure modes and what maintenance activities can cost-effectively prevent failures, resources can be applied where they will do the best. B. PREVENTIVE/PLANNED MAINTENANCE OPTIMIZATION Planned Maintenance Optimization is a methodology that focuses on improving maintenance effectiveness and efficiency by reviewing an existing maintenance program and, in most cases, adding maintenance tasks to account for failure modes not addressed by the existing program. Typically applied to Better-to-Well-Behaved Systems (20/80 systems-where only about 20 percent of all problems develop) are candidates for Experienced Centered Maintenance (ECM) analysis. ECM was adopted for the 20/80 systems to minimize impact of wastewater treatment plant/pumping stations personnel engaged in the overall reliability improvement initiative and reduce financial impact on the wastewater treatment plant budget for the improvement initiative. Each system requires a "quick look" -two to four days- by an in-house multi-discipline analysis team led by a specialist in facilitation that answers the following questions: defect elimination, root cause analysis, and safety, quality, and effectiveness. C. DEFECT ELIMINATION - ALSO KNOWN AS DON'T JUST FIX IT, IMPROVE IT Defect Elimination (DE) is a reliability strategy is to eliminate known defects caused by aging, wear and tear, workmanship, changed operating conditions requiring more robust components, or inadequate replacement parts that do not meet current stress levels present in an asset or technological advancements. DE analysis meetings typically can be completed in a day because they deal with known defects.Total costs less than $10,000 and work does not include 113 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 349 of 461 Page 116 of 195 replacement of an asset that cost$5,000 or more with a useful life of more than one year. Above these threshold values, the work would be part of a capital project. DE is typically a follow up action(s) from a PM (Preventative Maintenance), PdM (Predictive Maintenance), or CM (Corrective Maintenance) task. • Triggers for DE: o Optimization idea o Failure before asset useful life o Repeat failures showing up on the Bad Actors' list o Multiple reactive or corrective work orders on high-criticality asset o If more than 1 reactive work order with priority of 1 or 2 on specific asset between PM tasks DE helps reduce the amount of reactive work and reinforce precision maintenance. A DE work order template is based on the Don't Just Fix It, Improve It (DJFI) principle. Subject matter experts evaluate known defects in reliability, availability, maintainability, and safety of an asset. They then develop a control plan to eliminate the defect, review cost-saving opportunities, and ensure clear communication to all staff on the results/findings and any adjustment to the material, process, and people. Staff was able to complete 17 DJFI work orders in 2021. 114 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 350 of 461 Page 117 of 195 Our Plant Maintenance staff is constantly rehabilitating our assets to extend . their useful life and optimize their performance.They make it a regular - practice not to just perform the preventive maintenance (PM) or repair, but r also to ask what more can be done. Work orders that are categorized as"Don't Just Fix It; Improve It" (or DJFI) typically meet one or more of the following criteria: • Proposal of an optimization idea • Failure before an asset's useful life • Repeat failures showing on the Bad Actors' list • Multiple reactive or corrective work orders on a high-criticality f asset _ r0 More than 1 reactive work order with a priority of on specific asset between PM tasks From 2022 to December 2023, 33 DJFIs were completed.Some of these included the following actions: • Installed furnace burner signal splitters to allow the Plant better range of burner temperature control for efficiency and less natural gas usage Installed a new probe controller in the Aeration and Nitrification Tank for increased reliability, improved accuracy of dissolved oxygen measurement, and lower energy use through improved control of blower operation Installed a new wire harness in power distribution cabinet from circuit board to lamp rack connector Added a two-branched strainer system,which was designed and created in house, to the inlet now pipe of the water softener _ 0 Upgraded the lube delivery piping to the centrifuge element bearings • Designed and fabricated swivel brackets for water level sensing transducers for the Primary Sedimentation Tanks and the Forebay • Designed, fabricated, and installed rain shields switchgear at the ti Filter Plant from weather conditions Designed and fabricated a vacuum collection system to collect debris from the filter housing on the standby generators • Fabricated spare blowout panels to save time during clarifier PMs Figure 56:Central San's"Don't Just Fix It,Improve It"Maintenance Culture 115 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 351 of 461 Page 118 of 195 D. EXAMPLE - GREASE GUN COLOR CODE MARKINGS Central San Plant Maintenance utilizes different types of greases and needs to avoid mix-ups, using color coding of different greases to effectively reduce the chance of mixing different greases. A. � I - I. Figure 57:Grease Gun Color Code Markings E. EXAMPLE - RETURN ACTIVATED SLUDGE PUMP SEALS Plant Maintenance had issues with Return Activated Sludge (RAS) pump seals that had short mean time between failures. After an in-depth review and a pilot program, various changes were implemented to improve reliability, energy consumption, safety, and the PM SOP. An example of the change is hard chroming the shaft to eliminate the need to rebuild the seal when making adjustments. F. ROOT CAUSE ANALYSIS All the contributing and/or latent factors that result in failures occurring in a system lifecycle, especially those involving humans. Poorly documented maintenance processes, procedures, 116 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 352 of 461 Page 119 of 195 and work instructions all can be contributing causes to failures even after the most thorough Root Cause Analysis (RCA) is a process that is applied when situations or problems occur that demand attention to mitigate or eliminate them and there is uncertainty about all the contributing causes. RCA is needed because Reliability Centered Maintenance (RCM) and Planned Maintenance Optimization (PMO)/Experienced Centered Maintenance (ECM) can't be expected to identify CM or ECM analyses that are not completed. New personnel, lacking proper training and experience and rushed into a crisis, can overlook critical steps in completing repairs, resulting in infant failures shortly after resumption of operations. Changes in replacement parts, lubricants, and other items from the organization supply chain can be causes contributing to a failure. Changes in operating conditions and system configuration or practices can also be contributors. RCA investigations may take several days and the report of findings and recommendations for corrective action can be lengthy. ....................................................................................................................................................................................................................................................................................................................................................... I. EXAMPLE - ROOT CAUSE ANALYSIS INITIATIVES Central San staff formalized and implemented DE, RCA, and PMO initiatives in the following years,with their results tracked against assets in the CMMS and GeoPortal. Improvements were made through participation among different groups such as Safety, Regulatory, and Reliability Engineering. A 2021 example of an RCA was when an "atmospheric river" storm forced an abnormally high influx of grit-ladened dewatered sludge to enter the Multiple Hearth Furnace. This caused an Opacity Excursion resulting from unstable combustion. After identifying the root cause, a revision was made to the furnace SOP for the wet weather events to prevent this issue from repeating. G. SAFETY, QUALITY, AND EFFECTIVENESS OBSERVATION Safety, quality, and effectiveness (SQE) is an internal process to routinely review PM program based on observation of the PM or PdM Maintenance task. H. RISK THRESHOLD INVESTIGATION Risk Threshold Investigation (RTI) is used for 20/80 Systems that may never be subject to Reliability Centered Maintenance (RCM). An even shorter (one day or less) alternative analysis for 20/80 well-behaved systems is called RTI. The basic goal of RTI is to qualitatively identify potential problems or failures in 20/80 (well-behaved) systems that could: • Realistically occur and produce or affect one or more of the seven consequence areas (safety, environment, downtime, operations, regulatory requirement, single point failure, and economics). • Cross a threshold that is judged to be sufficiently severe to warrant special attention to mitigate or eliminate risk. • Lead to further analysis such as RCA, DE, PMO, or RCM based on RTI levels. 117 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 353 of 461 Page 120 of 195 The RTI process, utilizing the experience and judgment of a team of experienced technicians and/or first line maintenance supervisors and an operator does the analysis. Analysis is facilitated by a specialist in RTI methodology conducting the five-step process. These steps mimic, to a certain degree, the analysis employed in the classical RCM studies but stop far short of the comprehensive information collected and deliberation conducted in classical RCM. The whole RTI process typically takes less than one workday. Routine periodic review of Central San's Preventative Maintenance (PM) program is based on a strategic plan to perform reliability engineering (RE), RCM, PMO, or RTI analysis per two-year period. The RE analysis is determined by the 80/20 Pareto rule. I. FAILURE REPORTING ANALYSIS AND CORRECTIVE ACTION SYSTEM Central San's maintenance activities are primarily proactive but having a formal process to respond to and prevent future asset failures which ensures continuous improvement of equipment maintainability and reliability. Assets can fail in many ways: capacity, performance, efficiency, and physical mortality. Because of this, a process to control and mitigate these failures is invaluable. A significant change to the maintenance program's strategic framework was the implementation of a Failure Reporting Analysis and Corrective Action System (FRACAS). Implementing this system was a logical next step in Central San's maintenance program. Having already implemented the foundational elements of an effective FRACAS, such as the asset hierarchy, criticality analyses, and piloting RCM analyses on various systems, it was clear that FRACAS would be a welcome addition to Central San's maintenance program. Although Central San's maintenance activities are primarily proactive, having a formal process to respond to asset failures and prevent future failures ensures continuous improvement of equipment maintainability and reliability. Note that the Weibull Analysis is to be implemented. The workflow diagram is shown in Figure 58. The daily activities for best practice framework are best illustrated in Figure 59. 118 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 354 of 461 Page 121 of 195 Failure Reporting, Analysis, and Corrective Action System (FRACAS) Workflow Diagram Reactive Work crisis Emergency/ create Complete Repair/ After Action Order hornic Urgent P WO in Replacement Analysis - RCA 0 Risk Threshold Review Pareto: Investigation Perform BA Review Create RTI Appropriate Complete Result/Action Close out Complete Child Chronic Review Work O in JBD) Analysis; Repair/ Items and Create Chi 4 Order History f,Af%A eibull RCfA or PMO Replacement WO if needed Analysis JBD) Chronic Corrective or Part of normal Inspection/ DE or QE Complete Repair;' I Work order Planned PNI Replacement Acronyms: A. Work Order(WO) B. Computerized Maintenance Management System (CMMS) C. Root Cause Analysis (RCA) Figure 58: Failure Reporting Analysis and Corrective Action System(FRACAS)Workflow Diagram D. Bad Actor(BA) E. Risk Threshold Investigation (RTI) F. To Be Determined (TBD) G. Reliability Centered Maintenance (RCM) H. Planned Maintenance Optimization (PMO) I. Preventative Maintenance (PM) J. Defect Elimination (DE) K. Safety,Quality,and Effectiveness (SQE) 119 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 355 of 461 Page 122 of 195 Reactive work orders minimized, Breakdown / Functional Failure majority of Maintenance Work Orders Work Orders are reviewed on a Step 1 : Step 6: Work Comes from PM/PdM Monthly Basis for Corrective Action PM/PdM Compliance/ FRACAS Identification Metric - 95�0 Metric - 100% of Reactive WOs 100% Safety and Title 5 Compliance/ follow FRACAS Workflow Process/# <15% Reactive to overall WOs of RCA completed Follow Best Practice Framework CorrecVve Maintenance an Plant Systems August 2019-August 2020 surymd.m w P—Whaft u b 'a d 56p,m¢m ss u m B All Work Orders are s ua�a� �g t 2' Maintenance Work is Ste 5. � i � � � � � � � � , � , , � , , � , . • . Step closed out with all codes p mm m m e h p planned " e ��r � � �F � �,� a �^ tw'oak 9` �e m am � � � a� ,�' a' c°° � Work Order �� � �� �$ s� a���� o d�� Fb e"o F° Maintenance and information in CMMS 20/80Systems Metric - >85% Planned Close Out Bad Actors Better to Well Behaved Systems Planning Metric - 90-100% WOs . FnnRonohva oamuenAna,y:pryrcMTeam �`"° WOs vs Unplanned •All likely Failure Mptles AnahRetl Esisang PMfPGM Aadls and Hrstoacal LM'sAnalysistor Validatim,Change orAtltlhivn of Failure Maigating closed accurately ApdiraNe and Mnrt al.'. ak TasFs Implemented Pert°rmazperrtrategic pHn,mmplefe SRE per2yearPaiotl WOs Perto m riper rtrneg aan,mmplere RTI(TB J s AF per zyea peaad •Qualitative-O&A95ubleR MatterEWer[Team Identifies P°taaial'Shwu S[opperY'Ihing]Risk Criteria • Pertartred nn ssmms upon whim Erinn�yn done RCA 5QE Defect Elimination(TBD) • Regulabry,Safery,Cnt,Protlurdon,lahaf Safety,quality,Elf-enryllntamal AarPtl Repeat Failures fi 1-1 urne.W Parts Time •Some Covered Wdhlob Hazard Analysis andfor Praaices •Goal Oriented Cause&EftR&Process Performed on large lobs,New lobs,and Routine Rary Mahnals,Operations,Wohmamhip, k.IpbTeam Audits Design,Culladral 0emage E—mm dm Needed • Perform Whar Wuselsl unceaain or h Perform as neetlptl Perform as part aliwNnewi¢imwas uidallry failure lmprovertent Maintenance Work is executed to specification following repeatabl ") procedures to optimize functional Step 4: Ste 3: p performance Work Maintenance Maintenance Work is scheduled Metric - <10% Rework WWI% WOs Execution Scheduling Metric - 100% Maintenance Labor include DE improvement/# of SQEs hours scheduled by day by hour/ performed/# of planner updates >90% schedule compliance completed/# of GIS updates completed/# of exceptions identified Figure 59: Daily Activities for Best Practice Framework 120 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 356 of 461 Page 123 of 195 J. RELIABILITY ENGINEERING Optimizing work execution has been essential to Central San's reliability journey. In 2014, staff turned to the Uptime Elements framework, Effective Utility Management guidelines, and the EPA's 10-step Asset Management process to identify, prioritize, and address the inefficiencies in Central San's processes and practices. In 2015, Central San hired a consultant to develop an Asset Management Implementation Plan to advance the efforts of the Asset Management Program. Staff's efforts led to Uptime Awards for Best Reliability Engineering for Maintenance Program in 2018 and Best Asset Management Program in 2020. Central San continues advancing its Strategic Plan, developing, and enhancing processes, investing in new technology and tools, and continuing staff training to shape strategy into practical implementation. The award-winning practices are evident through our motto, "The right maintenance, right equipment at the right time." Central San has a proven record of operational excellence that includes 25 years of 100% compliance with our NPDES permit—the longest successful streak of any wastewater agency in California and among the top 20 in the nation. Concurrently with the CMMS implementation and what would become the cornerstone of the maintenance strategy, Central San piloted a reliability centered maintenance (RCM) approach. The pilot was led by Anthony "Mac" Smith, a pioneer in RCM. The Plant Maintenance Division's goal was to formulate a systematic approach to developing a comprehensive maintenance program based on asset criticality and consequence of failure. As was found through the pilot, RCM is a qualitative decision methodology that identifies the most effective preventative maintenance (PM) task for treatment plant equipment and systems. Additionally, RCM with a CMMS establishes a repeatable program with documented processes and procedures. Prior to jumping into an analysis, the maintenance team spent a week with a consultant and their company discussing Central San's current maintenance practices and issues, then learning classical RCM methodology. The discussion covered details of the seven step RCM systems analysis process and case histories which included the strategy for the deployment and use of RCM, experienced-centered maintenance (ECM) analysis, defect elimination (DE), and root cause analysis (RCA). Table IS:Experienced Centered Maintenance(ECM)Analysis Results SubsystemsRCM Systems Analysis Profile Centrifuge Subsystems Waste Heat Boiler Subsystem Functions 6 7 Subsystem Functional 9 11 Failures Components in 16 25 Subsystem Boundary Failure Modes Analyzed 46 63 Critical 29 (63%) 28 (44%) Non-Critical 17 (37%) 35(56%) Hidden 13 (28%) 7 (11%) PM Tasks Specified 62 70 (Includes Run to Failure) 121 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 357 of 461 Page 124 of 195 Active PM Tasks 53 58 Items of Interest 30 16 Note:The details for the above topics are in the"RCM Worksaver"software that was completed by the RCM Team. PM Task Comparison (By Failure Mode) Centrifuge Subsystems Waste Heat Boiler Subsystems RCM task= Current Task 0 (0%) 11 (16%) II RCM task= Modified Current 21 (34%) 24 (34%) Task III RCM Specifies Task, No Current 29 (47%) 24 (34%) Task Exists IV RCM Specifies Task, Current 3 (5%) 0 (0%) Specifies Differ Task V RCM Specifies Run to Failure, 1 (1%) 0 (0%) Current Task Exists VI RCM Specifies Run to Failure, No 8 (13%) 11 (16%) Current Task Exists A significant number of the study failure modes have no PM task currently. About one-third (34%) of the study failure modes currently have PM tasks that agree with the RCM PM tasks (II), but these rely on tribal knowledge to assure that they are executed. Plant Maintenance developed the strategic framework,which would become the road map of Central San's efforts to optimize its maintenance program. After completing the RCM analysis, the organization piloted an experienced-centered maintenance (ECM) analysis on the primary process, ultraviolet (UV) system, and boiler feedwater system, all with successful results. The strategic framework identifies Central San's approach moving forward to perform RCM analysis on the 20 percent of assets that are "eating our lunch," optimize the PM program for the better-behaved system (i.e., the 80 percent of assets) by performing an ECM analysis and perform DE as the situation or opportunities arise. Also, at any time or as needed for new or complex tasks, perform an internal audit for SQE of the tasks. Finally, perform RCA on regulatory, safety, or high impact operational failures. Identifying ways equipment can fail was well understood after completing the RCM pilots, so another proactive way to eliminate them was created. To address failures caused by the five sources identified by the Uptime® Elements, a DE program was developed. This program would be used across the board on both well behaved and bad actors, as needed. Potential triggers for defective elimination were discussed and used, such as failure before the asset's useful life, repeat failures showing up on the bad actors list, when there's more than one reactive work order with a priority of "1" or "2" on a specific asset between PM tasks, or simply as an optimization idea. 122 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 358 of 461 Page 125 of 195 Figure 60:Before Optimization r _ Lm— ��yjJ)yj11/.xr+r:r:i••r+•.•.• risi�i�j!+■ �/1/yyJJIIl1JIl/l ll111:1lY■/Y7 Tr'/!•Il�gti�. h:+L Figure 61:After t Optimization, Smooth Transition Eliminates Buildup Points �OL L 1 123 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 359 of 461 Page 126 of 195 Reliability Centered Maintenance (RCM) is a qualitative decision methodology that identifies the most effective planned maintenance task for equipment and systems. RCM, coupled with a CMMS, establishes a repeatable program with documented processes and procedures. Staff developed system-block diagrams, analyzed the functional failures and failure modes of assets and their components, and used a logic-tree analysis to select appropriate planned maintenance tasks. In addition to learning the RCM process, the pilot teams gained a better knowledge of the assets and how systems could fail. ............................................................................................................................................................................................................................................................................................................................................. I. EXAMPLES OF PREVENTATIVE MAINTENANCE OPTIMIZATION Central San routinely utilizes Planned Maintenance Optimization (PMO) to enhance the maintenance program and improve reliability. In 2021, staff completed PMOs on: • Cogeneration • Auxiliary Boiler • Influent pumps/motors • Boiler feed water pump/turbine • Cake pumps-Switched the runtime to do rebuilds from 2,000 hours to 3,000 hours • Long Flight drive sprockets-replace on a 5-year runtime basis instead of annually • Switching from Gore-Tex brand polytetrafluoroethylene tape to Equalseal K. PREDICTIVE AND ASSET CONDITION MONITORING PROGRAM The Plant Maintenance Division also employs predictive methods, including oil analysis, vibration analysis, and thermography. The division implemented condition-based/predictive methods such as oil analysis and lubrication, vibration analysis, thermographic analysis, laser alignment, and cathodic protection surveys. Staff reviewed and improved condition-based/predictive methods to complement the existing program. Staff continues to include new predictive maintenance methods and employ new technologies to be more effective. In 2021, staff piloted Ultrasound Testing technology on steam traps and found that 25 out of 80 traps needed repair. The Ultrasound Testing Program is now part of Central San's Asset Condition Management Program. Staff are trained and certified in how to conduct the testing, Ultrasound Testing inspection forms were created in the CMMS, and the division developed a PM task for steam assets. The successful implementation of Central San's asset condition management program allows us to focus on the left side of the Specify, Design, Installation, Potential Failure, and Failure Safety Reliability Curve to improve maintenance efficiency. Central San has established a culture whereby O&M staff are integral to capital projects' planning, design, and construction phases. This culture has resulted in many improvements and ultimately identified the need for a Building Information Modeling (BIM) and Computer-Aided Design (CAD) standards that prove to be great tools for Engineers and O&M staff. The following Predictive Maintenance (PdM) methods were identified and implemented: • Oil Analysis and Lubrication Program • Vibration Analysis • Thermographic Analysis 124 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 360 of 461 Page 127 of 195 • Decreased Performance Analysis • Cathodic Protection Surveys • Laser Alignment o Staff trained in laser alignment procedure o SOP fully documented and implemented for more equipment • Machinery Lubrication o Seven technicians earned Machinery Lubrication Technician (Level 1) and one technician earned Machinery Lubrication Technician (Level 2) certifications (International Council for Machinery Lubrication) as of December 2023 • Vibration Analysis o Five technicians trained and certified in RDI Motion Amplification technology as of December 2023 • Motor Testing o Motor Winding Analysis implemented and baselined o Digital Low Resistance Ohmmeter o Megger meter testing o Protective relay testing o Alltest Pro 5 Motor Circuit Analysis now used in motor analysis • Ultrasound Testing o UE Ultra Probe 9000KT now used for ultrasonic testing • Infrared Thermography o Three technicians trained and certified in Infrared Thermography Technology as of December 2023 • Non-destructive Testing o Remote field testing used in condition assessments o Guided wave inspection for piping 125 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 361 of 461 Page 128 of 195 F F Infrared Thermal F'F'achinery 11 AAA Lubric Imaging . . k --i Fluid F Ultrasound 01 Analysis kh • Vibration Analysis Motor I Condition O� Information _ Testing Alignment d' Non- LK • • • ' LID Balancing Testing • Lo Figure 62:Central San's Asset Condition Monitoring Program Begins with Uptime Elements Recurring work orders are scheduled through the CMMS for Plant Maintenance Division staff to take oil samples based on run-time hours from specific assets and ship them to an outside laboratory for analysis or to take infrared photos of particular assets. Maintenance Planners review the results of recurring and predictive work orders to determine the optimal time to replace the oil and monitor the wear of the equipment. Metal contaminants in the oil sample can indicate the failure of specific components that could be changed before catastrophic failure of the equipment. Transformer Oil Analysis assessments will be conducted biennially through a technical services contract by the Plant Maintenance Division. Vibration analyses will be scheduled and conducted through a technical services contract by the Plant Maintenance Division. Regular measurements of rotating equipment reveal trends of increasing vibration so that maintenance is performed, avoiding equipment failure, and unscheduled downtime. Thermography can detect abnormally high temperatures due to equipment malfunction or loose connections. Thermal images of a boiler and furnace's exterior surfaces are also taken to detect hot spots caused by damaged refractories. 126 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 362 of 461 Page 129 of 195 L. PLANNING AND SCHEDULING Implementing a Central San-wide CMMS was a significant part of the Asset Management ecosystem. However, retaining the scheduling functionality of the previous CMMS proved to be a significant challenge as Central San transitioned to the new system. After working closely with the Asset Management group to address these issues, maintenance staff identified the following improvements to the scheduling process: • Expanding the role of the planner in scheduling work • Utilizing new methodologies to improve overall maintenance productivity • QA/QC of work orders Employing the methodologies outlined in Doc Palmer's Maintenance Planning and Scheduling Handbook was one way to accomplish these goals. Central San's maintenance planning group already excelled in securing parts/tools, scoping the work, and minimizing work delays for various jobs. It was clear that expanding the planner's role in scheduling work was a much-needed change to increase maintenance efficiency. Maintenance staff worked with Doc Palmer and the GIS group to facilitate this initiative to define necessary fields to sort through a work order backlog. Proper allocation of work orders from this backlog of a shop's availability results in a more efficient scheduling system.This frees up valuable time a shop supervisor could spend with their technicians for ongoing maintenance work and improving QA/QC of incoming work orders. Using a combination of tools developed in-house and Doc Palmer's scheduler, staff developed the following work-scheduling process: ----------------------------------------------------------------------------- Maintenance Planners can Doc Palmer's Schedule Lookahead access the Scheduler Program generation schedules are ' Export Tool sorts the Work tool creates a distributed to schheduledul e located in Orders based on l craft Cityworks and shop availability e with supervisors and export the and allocates these operators for Work Order scheduled review ; ' them accordingly work orders , Backlog , v , �•��i��lt�nrt7 ncr. llkir uirEL ---------------------------------------------------------------------- Figure 63:Work-scheduling Process 127 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 363 of 461 Page 130 of 195 M. MAINTENANCE INVOLVEMENT IN CAPITAL PROJECTS Another crucial element in Central San's maintenance program is having O&M staff involved in all asset lifecycle phases. It became essential to have O&M staff provide support in the Planning, Design, Construction, and Start-up phases of capital projects. This "design for maintenance" allows for significantly improved ease of maintenance and reliability of the newly installed system or equipment. • Planning o Identify scope items to include in a project o Safety enhancements: e.g., built-in infrared cameras for monitoring bus connections inside switchgear, remote panels for operation of low-powered circuit breakers remotely, arc flash-resistant gear, etc. • Design o Select the right equipment like smart MCCs or standardize equipment when feasible to reduce spare parts o Ensure the maintenance needs are not overlooked, such as access, lift points, and housekeeping elements • Construction o Review submittals o Review shutdown requests o Job walks o QA/QC of contractor work Maintainability And Reliability Figure 64:Operations and Maintenance Staff are Now Involved in all Phases of the Asset Lifecycle 128 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 364 of 461 Page 131 of 195 N. BUILDING INFORMATION MODELING Including O&M staff is especially important in the planning and design stages of capital projects. Poor execution of construction projects can result in major inconveniences for O&M staff, significantly limiting their ability to perform their job functions. 3D Building Information Modeling (BIM) software allows staff to pinpoint and minimize these design deficiencies before construction work begins, improve the efficiency of maintenance activities, and optimize construction sequencing to enable continuity of operations. Staff have been able to address the following issues using BIM: • Equipment draining and flushing • Repair, lifting, removal, and reinstallation access • Isolation or bypass of process options Staff is working on developing a Computer-Aided Design (CAD) and BIM standards to assist with better asset tagging and asset handover. The BIM Standards will ask consultants to provide asset position number, asset class, asset class type, and other attributes in the model. Asset management can leverage the database behind BIM to capture asset information in a more efficient manner. Figure 65:BIM Example:3D Model of Cogen Gas Turbine Area Versus Actual Layout 6. RECYCLED WATER DISTRIBUTION Staff prepared a Recycled Water Program Manual (Central San, 2006) to describe the implementation and administration of the program. The primary purpose of the manual is to provide the regulations and administrative procedures of Central San's Recycled Water Use Permit Program. The Use Permit Program was developed to ensure recycled water's safe and efficient use per the California Department of Health criteria and comply with the San Francisco Regional Water Quality Control Board's General Water Reuse Order 96-01 1. For PM, CSOD performs routine maintenance on the system, which includes: • exercising valves • sampling and testing for residual chlorine • system flushing 129 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 365 of 461 Page 132 of 195 Central San has a contract with the Contra Costa Water District for corrective maintenance to perform repairs on the system. 7. STANDARD OPERATING PROCEDURES In 2015, Central San's Succession Planning Committee identified the need for a Standard Operating Procedure (SOP) program consistent across the organization. Senior Engineer Dana Lawson assisted the Succession Planning Committee in developing a template for SOPS based on the Information Mapping principles used for the POD SOPS. The Committee worked with IT staff to set up Laserfiche (Central San's electronic document management system) as the central repository for SOPs, then provided staff training and tracked the uploading of SOPS to the system. A. PLANT OPERATIONS DIVISION In 2012, Senior Engineer Dana Lawson contracted with Information Mapping to update the POD's SOPs, where 225 SOPs were re-written in a standard format with consistent terminology and presented the information in a clear and easy-to-read manner for the Operators. The draft SOPS were turned over to the Plant Operations Superintendent to review and finalize with his staff. B. PLANT MAINTENANCE DIVISION In addition, during the CMMS implementation, Central San Project 8242, Senior Engineer Dana Lawson assisted Plant Maintenance with the updating of over 140 planned maintenance procedures and developed the Cityworks Custom Business Process Manual (Lawson, Cityworks Custom Business Process Manual, 2017). Additional updates or new SOPS are the responsibility of the Maintenance Reliability group. C. PUMPING STATIONS SECTION O&M Manuals were developed in May 2011 for each large pumping station (Martinez, Fairview, Maltby, Lower Orinda, Orinda Crossroads, Moraga, and San Ramon), including system descriptions, component lists, color schematics, operating procedures, and control strategies. 8. MONITORING AND MANAGING SYSTEM BREAKDOWNS A. COLLECTION SYSTEM Central San maintains a 24/7 call center for the public to report plugs, overflows, or other system concerns. One crew is kept on-call and dispatched as needed. Additionally, CSO maintains 17 manholes throughout the collection system with water elevation sensors installed on the underside of the covers. When the water level in the structure rises past a defined threshold, the associated system sends an alert to the CSO Supervisors, Superintendent, and Division Manager mobile devices. The on-call crew is then dispatched to the site. B. TREATMENT PLANT AND PUMPING STATIONS The treatment plant features redundant equipment and systems. This redundancy allows one unit to be taken offline for routine preventive or predictive maintenance activities or capital projects 130 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 366 of 461 Page 133 of 195 and provides additional capacity for wet weather events. The treatment plant always maintains 24/7 staffing with Operators on shift. While the SCADA systems provide Treatment Plant and Pumping Station Operators equipment control capabilities, it also serves an important role in monitoring if equipment is operating within its defined parameters. If they are not, an alarm will audibly and visually notify Operations staff of the issue. Staff can then resolve the problem within the SCADA system or dispatch staff to the site. After-hours pumping station alerts are sent to the on-call Pumping Station Operator via messaging service. C. RECYCLED WATER DISTRIBUTION Plant Operations staff monitors the system pressure while pumping. Significant deviations will trigger alarms on the Plant SCADA system, and POD staff will investigate them. If field repair is needed, a contract is in place with Contra Costa Water District for on-call services, and CSOD staff can use system valves to isolate the break to minimize the impact on other customers. D. GENERAL IMPROVEMENTS The Treatment Plant buildings are occupied during working hours, and staff create work orders in the CMMS for the Plant Maintenance Division if there are any issues. If the problem is urgent, staff will call the appropriate Plant Maintenance Shop Supervisor directly to report the problem. For Fleet vehicles, CSOD implemented an Automatic Vehicle Locator service in 2017 to monitor vehicle status and maintenance codes. The Vehicle and Equipment Maintenance Supervisor reviews the data daily and schedules corrective repairs or inspections, as needed, through the CMMS. For IT assets, the IT workgroup uses a "help desk" ticket managing software to manage IT supported equipment and system requests from staff. In addition, the IT Section implemented a 24/7 on-call service for after-hours IT support for issues that cannot wait until the next business day. 9. UPDATING AND MAINTAINING PROCEDURES During the implementation of Cityworks, a universal custom field was added to all work order templates labeled "Planner Update?". Maintenance staff must complete this field to notify the Plant's Maintenance Planners or CSO's CMMS Scheduler if the procedure, attachments, instructions, or cycle needs modification. The Maintenance Planners have an inbox showing all work orders with this field marked "yes." This feedback process has significantly improved the update of procedures. A. COLLECTION SYSTEM AND RECYCLED WATER DISTRIBUTION The SSMP is updated on a five-year cycle by the CSOD staff and by the Associate Engineer responsible for the Collection System in the Planning Group of the Planning and Development Services Division. Starting in 2018, the SSMP is audited annually by CSOD. The SOPs are updated by the CMMS Scheduler and Administrative Technician when CSOD staff requests and the Supervisor approves. 131 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 367 of 461 Page 134 of 195 B. TREATMENT PLANT, PUMPING STATIONS, AND GENERAL FACILITIES Table 16 designates frequency and responsible party for updating and maintaining procedures. Table 16:Procedure Update Owners Procedure (quantity) Responsible Party Last Updated Freq!��y Hazardous Energy Control Electrical Shop Supervisor Unknown As needed Procedures 800 Operations-SOPS (225) Plant Operations 2012 As needed Superintendent Plant Maintenance Procedures Maintenance Planner Most in 2016 As needed (PM Plans, 300) 10. PERFORMANCE MONITORING AND REPORTING Access to information was crucial for decision-making during the implementation of the Asset Management Program, and it continues to be an important resource in Central Son's day-to-day O&M. A variety of reports are accessible through the CMMS report list, including those seen in Table 17. Users can generate these reports for a custom-selected date range. Table 17:Computerized Maintenance Management System Reports KPIs PM, T5, NPDES, Safety Work Order Compliance KPIs Last 12 Months PM, T5, NPDES, Safety Work Order Compliance Last 12 months Assets by Process Full assets list with condition scores by process RM by Process Pareto- Reactive Work Order by Cost and Count by Process RM by Sub Process Pareto- Reactive Work Order by Cost and Count by Sub Process Child Work Order from PdM/CbM Exception Report PdM Last 12 Months Bad Actors Bad Actors by Process Assets Condition Condition Scores Spares By Asset detail of parts needed for each asset and warehouse stock on hand Reserve Assets New additions to the asset registry as capital projects are completed Backlog By shop Overtime By shop Closed Work order type distribution: Reactive, Corrective, PM, or PdM DE/DJFI Work orders with types: Energy Cost Savings, Improve Resiliency, Simplify Operations, and Staff Time Savings 132 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 368 of 461 Page 135 of 195 Supervisors and technicians perform workflow monitoring using a variety of "inboxes," configurable and displayed on the user's home screen in the CMMS. Table 18:Workflow Monitoring Inboxes Many of the TP Maintenance Inboxes customized to the craft level TP schedule busters TP work last four days TP open work orders TP work orders on hold TP Weekly Schedule TP weekly schedule by shop TP weekly schedule next week TP in Progress In Pro ress Graph TP Annual Major Maintenance Major maintenance Furnace turnaround Multi-shop Work Order by shop TP KPI T5 almost due T5 past due All T6 Safety almost due Safety past due All Safety PM almost due PM past due All PM Implementing this new monitoring and reporting framework allows greater visibility, leading to future maintenance efficiency improvements. Next are examples of performance and monitoring reports. 133 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 369 of 461 Page 136 of 195 - — ` CENTRAL SAN Bad Actors _ I CENTRAL SAN Reactive Maintenance Work Orders • • for 10/1/2020 thraugh 10/1/2023 by Praceas • • Fro m 1011/2022 to 10/2/2023 RM Wo %R1d WO Consequence Possibility of Business Risk Asset ID Legacy ID Court RM WO Cost Asset Cort CoAIAssetCast Corditicn Canditim Date Install Date Service Life of Failure Failure Exposure Process Sub Process system Entity Type 10 10WU Headquarters 7 $1,919.13 $3,WU,DWAO 0% 30.00 201408-00 1988-05-01 75 1 3 3 Buildings Headquarters structures Building Office Building(HOB) Office Bldg pr(lmy5 Closed REYl1Me WD OWlt 7RM CAPI 1595 5I141 Gale 22-Man 4 $82,Cf100.00 2% 2019-114)1 40 1 2 Tr nnem Flam Roads& G"P6 fJull�n�a la 3,9r5.J0 Gale now QUIP. Parking Dwril-Lion$Fiud Em•uertt lrl 3'aw.3r- 2670 72101 Furnaces 6 5116.80 $1o,W0,OW.00 0% 65.00 20151&30 1983-0418 30 4 6 24 Wi&Treallu m Incineration Fumaces F.— Finer MnT&aeW 2 N'4S9.w 3021 74004 C-ni 6 $1,43044 5149PWA0 1% 1999-05.20 20 S 30 Wids Treamrerrt Sludge CentnIl a Centrifuge fi8nergl 1 244.iN Pos@icn 1 De-tiering Primary 7—Ttri 7.9 9,803.43 3184 74415 Continuous 4 $1,411.47 $I,DWAO 141% 80.00 21408-08 199 "i 20 9 8 72 5aids TreaYnenS Incineration Fumaces Instrumem PumpStltion 1 14,488.88 Eir isslm N%arn or lCEM} .1. 5eDOn%ry TrlaBbllerlt 13 14,234.1d 37,97 96000 Household 6 $1,898.42 $5,797,W0.00 0% mml2D14{18-08 1997-07411 6O 1 2 2 Buidngs Household structures Building aiow rk 4 1,617.M. Fazardous Waste hazardous Waste Sands TTc alwm 34 2E,844.U1 (HHW)Building Bldg Support 3ysiems 1 64.9d 389 22204 Long Flight 5 $1,181.35 $LfiRXA)0 7% 10.00 2018-09-23 2012-0G12 20 5 1 9 Primary Primary Pri Sed Basin Gearbox Cd lector 4 DF- Treatment sedimentation LIillti S 19 SIM." 5537 74140 Auxiliary Boiler 2 7 $1,151.54 $248,0W-00 0% 20.00 201408-M 1984119,01 35 8 Otilities Steam AuxB I.- Boiler 'Nat Waath-FanittluG 2 1,571.22 ir Generation 780 33101 N Clarified 5 $1,023.66 $248,000A0 096 50.00 2015-10-30 20C2-IMI 2C 1 4 28 Secondary Clarifiers Clarifiere Gearbox Cd lecor Drive Treatment 909 35113olaboramry(Lab) 0.70 55,osspim.UO 0% 20.00 21314-08-08 2001-CMG 60 7 2 14 Buildings Laboratory Bldg structures Building Reaetlue Maintenance Work Order Ciount and Total Cast by Process g � $32.d60 � $2Rd60 Figure 66:Performance and Monitoring Example: Bad Actors a 51k db0 - April 2025 20 r— ----------- ---- ----— 51a11Do CENTRAL AI+I np' �` 15 _�. Campllance,Safety and PMBPDM Work Orders For 1 ON1r2022 through 10IO2r'2023' '=Y u • k •_x Building d rieuirical Inslrumiert Machine Mair!er arwe Mdthonkil Plrnp SLIM - TOLalB& KPl r Grown& Reliabililk A4era.es fdru" ° ''• wrl �� r SG Kill nPi SLSIYeedoO 'her;1—& General F'I-N P—m Adm Si-vndry Mtikrk iaWs Support Ihtllrlen WnWwC,e ( C� 1i'1 9 FPUI Ii Tire— r—trirrr. Treaunrm i5—' F.Ildos Cum l04od drl-True .4 17 94 ' 7 141 Nx 2" w EtRYmt Told Compli 4 17 95 9 t47 Nx sae ,, 7 wark aw.w 7.runt O Tnw-1 ry Fmrx,rs Lti Cgiril loll On-Tole 10G35 104; +. is 7&34 iC644 cx. -r=c V% fw D13SH 7 I r. CunpwoedOn-Trn6 6 2 SG W 2 6 Ne. a1 °4°Y° �� p dI4E! NV" Twee u. Figure 67: Performance and Monitoring Example: Reactive Work Orders by Process Iii czmpli a 2 JE - 5 6 NA # Yat ny.,lr.T %Card .bed OrrTm6 7Sk IN% 89% 0% 406 10% 0% W% M Mry 2�29 C'17fI1 kmd On-Trod 45 NA 1 N4 NA NA NA is nr Fcw T0101 Q'nil?1f11f+11 15 NA 1 N5 NA NA NA U 9FQxnpk69dOn-Trna loran 0% 100% 0% 0% nx non 113001 0% r-a r:x CkrripJl Or-1 HA NA 11 N4 NA 1 NA 21 . �a. T1114 CopIPk'1L14 kh NA 11 Nil NA 1 NA 12 a "` 55Cam i9lrllma D1L 8% 103% D% ox 111118% ox 11113INF t79S g 4- N � sr Gcrrple4etl Url-Irn6 13 2 sJ NA NA 13 NA iid r-n Told C-ampli 19 A a$ W NA 19 NA 1U ,'• '7iCamplaw fin-Iims MY 1Dx sale U% ups 100% 0% !7% J12PM RV or Pill BLf k•/tl15 d3:d PT9C —M ru+PoY L+FGnf IRrL W TIIIG p rill Will On Triw 1,147 154 584 39 12 739 51 2W 1UnII:Yryn7 101RI COMPIIRM 1.391 t74 5% 53 77 734 53 3DU June 2P23 4 C'ulhl 4n Trno 91+5 89% 9&'ai 74% 5594 9934 96% 97% sm ,aK SAFETY r•. Cu rl On-YFw 9 1 2 1 1 1 NA 12 eFr Iola Unpli 9 1 2 3 1 1 NA is IN Cam bed OrrTmB, 100% Im 133% 10@% -.v. W. 0% 94% JLM t C,crWkftd del-Trod t NA I ..1.. , W. NA 49 Tulr!GoIFIVbIod t NA I NA 50 94Con liOrrTm6 1[I:s. 0% 1]]a ox 96K lr5s '•* op to ❑YVlel Un-Irne 14 let. Y.:: NA 2" lk TD1A rmpi -5 11 br 1 ., rV. 2i0 Fa '7i Camoem on-I IML LIW,* .- 75 '.t:l' 1. 3,xy Figure 68: Performance Monitoring and Reporting Example: Regulatory Compliance,Safety, PM,and PdM Work Order KPIs 134 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 370 of 461 Page 137 of 195 Closed Work Order Count by Shop 14OO 1200 Building and Grounds ~ z loaa Electric 4 Instrument 5 Boo W ❑ N Machine O 600 Maintenance Reliability 0 400 Mechanical 200gL--Ime dirmillillbi Pump Stations i 0 Corrective Non-Maintenance Predictive Preventive Reactive WORK ORDER CATEGORY — Central Contra Costa unitary District PMs& PDMs Resulting in Child Work Orders Closed Work Order Count Corrective Reactive 29% oet 6.2i M N&OZ2 IM&22 JM.23 Fa mhir n Apr-23 Mnr23 Jun.23 Jk&L23 Au p943 9. m 3% 10 15 7f 1R 17 71 20 9R 77 15 77 19 rJt m wCu from PDM WiDs 0 0 0 2 0 3 1 0 0 1 1 0 W05w OAi ithChATCo-ractieeMs A CILWII aax 12.5% 0.13% t.9411i 3.113 O% H Men 5.3H 4.5% 0.17% 223 32fi 252 732 742 2ER 209 M 23E 22E 272 310 mina WD,fuwli phi M. 7 7 5 15 4 7 3 1 AA R A swnhtAdLffTacn,&viii 0.9% 09% Z0% 6.9% 1.7% 2.6% 1,47E OA% 1.57% IA% 2.9% L9% Non-Maintenance 6% Predictive Preventive 4% 58% Figure 69:Performance and Monitoring Example:Closed Work Order County by Shop t � PREDICTNE PREVENTNIE 4 7 29.. 0% _o —_� _ _ _ - _ _ ra CV cv Th ra f%ru o 4 Figure 70:Performance and Monitoring Example:PMs and PdMs Resulting in Child Work Orders 135 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 371 of 461 Page 138 of 195 1 1 . OPERATIONAL RESILIENCE AND DISASTER PREPAREDNESS As public employees in the State of California, all staff is "declared to be disaster service workers subject to such disaster service activities as may be assigned to them by their superiors or by law" (California Government Code, Title 1, Division 4, §3100). The Board adopted an Emergency Operations Plan (Deutsch, Emergency Operations Plan, 2016) in 2010. The plan is reviewed annually and revised as needed. The document provides an overview of the basic plan, the Emergency Operations Center (EOC), EOC position checklists, and the incident command system. Central San has designated and equipped the Multi-Purpose Room at the Martinez campus as the primary EOC. The main training room at CSOD is prepared to serve as a backup EOC and a sub-site for CSOD during an emergency. Emergency management training is provided to specific staff and Risk Management facilitates annual training exercises for the EOC. Exercises are conducted annually with key staff. Central San also participates in county-wide hazard mitigation plan development. It is a California Water/Wastewater Agency Response Network member to share resources in the event of an emergency. In addition, the following are available in the event of an emergency: • Crisis Communications Plan (Barnett, 2020) • Safety Directive 13.0 Emergency Action Plan (Ledbetter), October 2011 • Emergency Operations Plan (Deutsch, Emergency Operations Plan, 2022) -for use during activation of Emergency Operations Center • Sanitary Sewer Overflow and Backup Response Plan in Chapter 3 of the SSMP • Emergency Response Plans Wastewater Pumping Stations (Cortinovis, 201 1) Central San has backup equipment, including generators, at crucial points in the infrastructure, such as pumping stations, to ensure continuity of operations in the event of various interruptions. A. PUMPING STATIONS A few pumping stations have Mission M-80 floats to monitor the upstream maintenance hole in case of failure or problems. Contingency options for the pumping stations include bypass pumping or pumping the wet well of small stations with a Vactor truck, summarized in Table 19. Table 19:Contingency Options BypassStation . • Capability - Well Acacia x x Bates Blvd x Buchanan North Lift Station LS x x Buchanan South LS x x Clyde x Concord Industrial x Fairview x Flush Kleen x x Lower Orinda x Maltby x Martinez x Moraga x 136 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 372 of 461 Page 139 of 195 Orinda Crossroads x San Ramon x Sleepy Hollow x x Via Roble x Wagner Ranch (private) x Wilder Lower(private) x Wilder Upper(private) x Despite the various improvements made in maintenance to optimize the lifecycles of Central San's assets, external factors could still threaten the operational reliability of all facilities in the Central San service areas. These external factors include seismic events, fire risks, flooding, or extreme events and pandemics. Responding to and preparing for these events has been a significant aspect of Central San's strategic plan. B. PUBLIC SAFETY POWER SHUTOFFS Increased wildfire risks have resulted in more Public Safety Power Shutoffs (PSPS) by Pacific Gas & Electric in recent years, making Central San's pumping stations especially vulnerable. However, successful planning has allowed for the continued operation of all 18 pumping stations, even during PSPS. By developing and implementing protocols for power shutoff events and acquiring equipment for these facilities, such as portable generators and a diesel storage trailer, Central San has ensured that its facilities continue to operate reliably in the future, even during adverse weather conditions. �y v,1111 1 :a.il al3-A Gonoerd Pre-Threat Areas ra>3c �a Tier s-Elevated r 4�r f lr.J1, Tier3•Extreme Gl�armc,i, San .latex Fremc-n9 CENTRAL COMA COirx SANITArf GOTpKT Figure 71:California Public Utilities Commission Fire Map C. PANDEMIC RESPONSE Continuing to provide essential services during the COVID-19 pandemic has proven to be a significant challenge for many organizations, and Central San is no exception. Effectively and 137 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 373 of 461 Page 140 of 195 efficiently managing the situation required a comprehensive crisis plan, which included the following key features: • Enforcing the Centers for Disease Control and Prevention's COVID-19 guidelines, such as social distancing, face covers, and general hygiene best practices • Adjusting employee schedules and placement to reduce face-to-face contact while ensuring adequate staffing for onsite emergencies • Implementing engineering control technology and safety practices • Setting up action plans if an employee tests positive for COVID-19 or a work area becomes compromised Successful implementation of our COVID-19 Plan has allowed Central San to reduce the risk significantly to our employees, especially those in the Plant Operations and Plant Maintenance Divisions, while still providing high-quality services to our community per our vision, mission, and values. 138 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 374 of 461 Page 141 of 195 DEVELOPMENTSTAFF An effective and resilient Asset Management Program relies on the strengths of the staff to be well-equipped to perform their jobs. Staff development and training is an integral part of continuous improvements. As of December 2023, Central San has had ten new Certified Reliability Leader certifications, two Certified Maintenance Manager certifications, with more staff involved in Certified Reliability Leadership workshops. In concurrence with the development of Central San's Asset Condition Monitoring program, staff has since received eight new Machinery Lubrication Technician certifications, three new Infrared Thermography Technology (Level 1) certifications, and five new Motion Amplification Technology (Level 1) certifications. For the Collection System, staff are certified by CWEA with Collection System Maintenance Certification. As of December 2023, these staff have the following: 14- Grade 1, 21- Grade 2, 5 - Grade 3, and 6 - Grade 4. Outside of training leaders, a culture of reliability should be developed at all levels of the staff. It has been challenging when recruiting and retaining skilled technicians who can operate and maintain an industrial complex. A primary cause for this challenge is a shrinking pool of newly qualified technicians to replace the retiring workforce. A second and substantial cause is the inefficient allocation of resources. To address this inefficiency, Central San committed itself to ensuring staff acquires the knowledge, skills, and leadership needed to achieve their goals and the goals of the plant to a high standard of quality and reliability. 1 . PERSONNEL ADVANCEMENT Opportunity for advancement is essential to the employee. For this reason, promotions and personnel advancements are made from within the workforce wherever practicable. A promotion shall result in a pay increase of at least one full step greater than received before promotion, taking into account any registration or certification differentials received before the promotion. Central San has a provision that provides employees in positions within designated job families the opportunity to promote to a higher class, without regular recruitment, upon meeting specified performance and testing criteria. An example of a job classification series is Maintenance Technician 1, 2 to 3. For other classifications, an employee may progress in the series if they meet specified performance and testing criteria and may only go to the highest level in the series if a vacancy occurs at that level, such as from Maintenance Crew Member 1 to 2 to Maintenance Crew Leader. Table 20:Job Classifications • • - Collection System 0 Collection System Maintenance Operations 0 Collection System Operations Professional Engineer Plant Operations 0 Wastewater Treatment Plant Operator Professional Engineer Plant Maintenance 0 Certified Crane Operator Electrical/Instrumentation Technologist • Mechanical Technologist • Plant Maintenance Technologist Professional Engineer Pumping Stations Operator 139 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 375 of 461 Page 142 of 195 Engineering anPlanning and 0 Professional Engineer TechnicalDevelopment Services Services Capital Projects 0 Professional Engineer It • Land Surveyor Environmental and 0 Chemist Regulatory Compliance 0 Environmental Compliance Inspector • Hazardous Waste Operations and Emergency Response • Professional Engineer Personnel Empioyment Cycle Step 7: I'll"hires are moru"e to Central Personnel are trained to replace Promotion and San and are onboarded by promcled and retiring employees Retirement Human Resource and Metric-Average time to fill Conmunwallon Servlcea vacancy f}m 25%irternal Metne-9eterm in each area)9 00 r romotiats career Palm aktende(I r% Ste 1: completed onbcardln� Step 6: P Succession Planning Recruitment and Onboarding Traift i ome s provided to persl ko prepare them for supervisory roles Metric-EfFeclive use of General Strategic Goal Metric-3{0€pmaleted Manager ataff ais and Bross Workforce Development. training and develoome� training . 4b staff nendance in leadership aoademy.supervisor Recruit, empower, and prCgram and meirtprship pn;!�ram Stop 5: engage a highly trained Step : areerAdvanoement Shop Training and and safe workforce Development Checklist Nmonnc retelve annual safety and ladMal training to loam ane reinforce skiiK Melm-%slaffCerhfiee Reliability Leader Stop 4: orSomty for Murmname &Rmabmy i i l Training Step 3: Professlonals cernfiee 1%staff CaWrn,a 9 Annual Performance etdrr l ro annual h e@ a kruel" V4Mr Environment Ascot rotation certiled,% Activities f staff As setConawn Monitorinpeerhfie(]I% Appraisals and Career Melne-%PerfonnarimAporalsals staff safelytra.ting canpllance Development Review emylaudon t ne l&wD safely commlaae oartlelo lon Figure 72:Personnel Employment Circle The Personnel Employment Cycle shows how Central San meets the strategic goal of workforce development and knowledge management through succession planning. This starts with recruiting and onboarding new employees, developing employees' skills using a Shop Training and Development Checklist, reviewing annual performance appraisals and career development, and providing training activities for career advancement. Personnel training and in-house skill development are encouraged for succession planning to retain system knowledge from retiring and promoted employees. 140 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 376 of 461 Page 143 of 195 CENTRA�CONTRA COSTA UNITARYQISTRICT CENTRAL CONTRA COSTA SANITARY DISTRICT PLAN T MAIN TE HAN CE DIVI S ION PLA N T MAaN TENAKC E DIVI SIO N MECHANICAL SHOP MACHINE SHOP MAINTENANCE MACHINIST TRAINING AND DEVELOPMENT CHEGKLfST T'RAfi�V��JG ANp r�$VaOPMEPdfEi4 7 CNlC1AN flFlfCKrJS7 y Figure 73:Training and Development Checklists for Crafts 2. EDUCATIONAL RESOURCES Successfully operating and maintaining a treatment plant of this size is no easy feat. Because of this, Central San has ensured operators and technicians have access to necessary resources. GeoPortal serves as a r central repository of information for assets ` and documentation. In addition, Central ' San's Plant Information Management System affords staff access to training materials, - O&M Manuals, and SOPs. Online training resources include TPC Training, Lorman PU%Ni PRCCEsSES Education Services, and Red Vector. Many of these resources are available through portable devices like iPads, which are provided to all operators and technicians. Figure 74: Plant Information Management System Main Page 141 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 377 of 461 Page 144 of 195 3. EXPANDING THE INTERNSHIP PROGRAM Central San's paid internship program provides students from various colleges with the opportunity to apply their academic preparation to practical work experience in a wastewater utility. Throughout the years, Central San expanded the program's scope and offered internship positions in the Administration, Engineering, and Operations Departments and the Environmental Laboratory from not just four-year universities, but those in technical and/or vocational programs. This internship program successfully provides valuable work experience to students and develops a pool of qualified, interested candidates for future work opportunities within the agency. However, the Plant Maintenance Division still faced a major challenge: a retiring workforce with a shrinking pool of new, qualified technicians to replace them. To address this shortage, Plant Maintenance management began a program to employ student interns from local technical colleges in the shops. This program began with Central San hiring two interns and three temporary employees for the Mechanical Maintenance and Buildings and Grounds shops. Student interns assist the shop's Journeyman Industrial Maintenance Technicians in various preventive, predictive, repair, and fabrication jobs on equipment throughout Central San's wastewater treatment plant. The interns gain work experience in an industrial plant while being mindful of safe working practices. Interns also learn the following skills with supervision from Maintenance Technicians: • Lockout/Tagout • Confined Space hole watch • Forklift operation • Use of a CMMS • Precision measuring • Pipe fitting 1 i _ r Figure 75:Plant Maintenance Staff Developing Central San's shop internship program remains an ongoing effort, with the intent to expand the program to other shops, such as the electrical, instrument, and machine shops, in the future. 4. MENTORSHIP PROGRAM Central San's Mentorship Program pairs employees from across the organization with a Division Manager or member of the Executive Team, and in some cases, Technical Experts. This program 142 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 378 of 461 Page 145 of 195 allows staff to boost their career as mentors or mentees. The program is not a traditional mentorship; it combines the development of personal relationships with project management experience. The mentee is challenged to complete a project that develops and showcases their skills as a mentee. Each mentee presents their completed project to the Executive Team, program peers, and the Board. While training courses and conferences can be valuable for staff development, there is nothing like receiving personal support and encouragement from mentors within Central San who may provide career growth insights. A technician who participated in the program said, "I've worked with several people that inspired me to be better. The Mentorship Program was a great way to work directly with these some people and gain knowledge, skills, and confidence to improve my performance here and possibly inspire the next person to do the same." 5. EXTERNSHIP PROGRAM Central San hosts a week-long, unpaid Externship Program for students from colleges, universities, vocational, or technical schools all over the country. Our cohort-style learning experience is designed as an insider look at multiple career paths in the wastewater industry and an opportunity to make connections, not only with established staff at Central San, but also with fellow externs. The program is free and highly customizable; it can include any specific careers or departments the externs might be interested in. 6. MANAGEMENT AND SUPERVISORY ACADEMY The Management Leadership Academy is a leadership program designed to help managers and supervisors improve their overall management effectiveness and provide opportunities for personal career development. The program takes place over an eight-month period and typically accepts up to 24 students (from Central San and sister agencies) who must complete an application. Once accepted into the program, there will be pre-session work, reading, class assignments, and a 360-degree review completed for each participant assessing current leadership effectiveness. Using this personal feedback, participants will refine their skills as they proceed through the various modules of the academy. Since the 2019 session, Central San has invited staff from other agencies to participate in the program as well. This cohort style ensures a well-rounded program with opportunities for participants to network and share knowledge and experiences with industry peers. The Supervisory Academy covers core supervisory competencies such as team development, performance management, coaching and counseling, labor/employee relations, conflict resolution, and effective communication. The program consists of seven to eight day-long off-site sessions. The goal is to allow the participants to understand the roles and responsibilities of effective supervisors, assess their skills and strengths as they relate to the core competencies, and set individual goals for professional development in supervision and leadership. Each session incorporates self-assessment, interactive exercises, and case studies to apply these skills in real-world situations. 143 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 379 of 461 Page 146 of 195 7. CAREER DEVELOPMENT PROGRAM A new program allows staff to create and follow through on a plan to reach career f � goals, whether inside or outside their current workgroup. The Career , Development Plan identifies staff goals, i4i; J actions, and resources needed to accomplish these. Human Resources pairs ti the staff with a Career Coach to explore that career, understand the requirements, , obtain training and education, perform job shadowing, and gain hands-on i experience. Additionally, Central San launched Employee Recognition Events program to i allow managers to acknowledge extraordinary achievements. Central San _ developed an Innovation Recognition ' Program, which included the first annual Innovations Fair, showcasing and celebrating innovative employees going above and beyond to improve and optimize their work. 1 CENTRAL CONTRA COSTA SANITARY DISTRICT Figure 76:Career Development Program 8. PERFORMANCE RECOGNITION, APPRECIATION, AND BEYOND Periodic internal and external publications announced and recognized the staff's hard work and outstanding effort. Appreciation lunches and rewards such as jackets, shirts, and hats were arranged regularly to continue the DJFI (Don't Just Fix It, Improve It) cultural shift. In addition to DE (Defect Elimination), staff focused on applying Total Productive Maintenance Sort, Set in Order, Shine, Standardize, and Sustain principles throughout the plant, such as maintaining a sorted, clean, labeled, and well-organized workplace. Primarily, staff listened to employees and followed up on employee suggestions via planner updates, GIS updates, and other ideas for improvement. Employees seeing their ideas and suggestions in action is a powerful way to show recognition and appreciation of employees' ideas and efforts. It also instills a culture for the pursuit of maintenance and operational excellence. 144 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 380 of 461 Page 147 of 195 Li Figure 77:Lubrication Room Figure 78:Monthly Internal Publication,"Lateral Connection" 711 kt _ Figure 79:Mechanical Shop Figure 80:Tools Room 145 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 381 of 461 Page 148 of 195 9. TEAM ACCOMPLISHMENTS The results of Central San's continued dedication to staff development showed the team achieved many accomplishments since the development of the new maintenance program. • 2022 NACWA Peak Performance Platinum Award: o Top 20 across all NACWA organizations (25'" consecutive year) • 2020 Uptime Award for Best Asset Management Program • 2019 Utility of the Future Water Environment Federation's Technical Exhibition and Conference • 2018 Uptime Award for Best Reliability Engineering for Maintenance Program • 2018 Cityworks User Conference "Excellence in Departmental Practices" Treatment Plant • CWEA Awards: 0 2020 Community Engagement and Outreach, Project of the Year, State o 2019 Mechanical Technician Person of the Year, SF Bay Section o 2019 Andy Stamatelos Special Service Award, SF Bay Section 0 2019 Collection System of the Year- Large, SF Bay Section o 2019 Community Engagement and Outreach - Best Use of Social Media Award, SF Bay Section o 2018 Engineering and Research Achievement of the Year, State 0 2018 Mechanical Technician of the Year, State 0 2018 Gimmicks & Gadgets, State o 2018 Photo of the Year, State o 2018 Electrical Instrumentation Person of the Year, SF Bay Section 0 2017 Research Achievement of the Year 0 2017 Mechanical Technician of the Year o 2017 Gimmick/Gadget of the Year o 2017 Photo Contest o 2016 Plant of the Year o 2016 Safety Plant o 2016 Gimmick/Gadget of the Year o 2016 Community Engagement & Outreach Project of the Year o 2015 Collection System Person of the Year, SF Bay Section o 2012 Large Treatment Plant of the Year o 2013 Collection System of the Year, San Francisco Bay Section o 2013 Supervisor and Collection System Worker of the Year, San Francisco Bay Section o 2012 Collection System of the Year, San Francisco Bay Section o 2012 Treatment Plant of the Year, San Francisco Bay Section o 2011 Large Plant Safety Award o 2010 Film Festival Award -Semi-Professional (Fats, Oils & Grease Video) In summary, a culture of reliability should be developed at all levels of the staff. The evolution of Central San culture translated into advancing asset management. Central San understands that effective leaders in a reliability program could be anyone or everyone, provided staff create a new future by eliminating defects, reducing the total cost of ownership, and supporting the organization's objectives. Central San has focused on engaging and empowering staff through leadership academies, advancements, mentorship, and celebrating successes. 146 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 382 of 461 Page 149 of 195 2023-2028 ASSET MANAGEMENT PLAN ELEMENTS 1 . LEARNING TAKEAWAYS Since 2014, Central San has continuously identified areas to improve reliability, instituted and developed tools and resources, and developed a strategic framework. Along the way, staff found the following: • Every plant has its specific maintenance and operational needs. Determine what will work based on Central San's discussion about its journey and the components of a comprehensive maintenance program. • Use condition assessment results to guide capital project decisions. • Encourage staff development-continuous improvement begins with capable staff and effective leadership. • Ensuring continuity of operations is more than just maintaining equipment. Developing action plans is necessary to keep facilities operational during natural disasters, pandemics, and other external events. Central San's asset management journey is a story of continuous improvement, from Central San's pen-and-paper maintenance system in the 1970s to its first use of CMMS software in the late 1990s to employing a comprehensive, GIS-centered Asset Management system today. From the 2014 workshops to identify a gap and needs analysis, staff implemented the following: • Piloting RCM (Reliability Centered Maintenance) analysis • Central San-wide CMMS system • Implementation of asset handoff protocol • Asset registry • Asset criticality • Asset assessment • Succession planning • Staff development • Systematic failure analysis With many of the above items addressed since then, staff identified the following things to build upon the original Asset Management Implementation Plan, make further improvements to Central San's maintenance practices, and ensure continuity of operations: • Formalization of Defect Elimination (DE), PMO (Planned Maintenance Optimization), and RCA (Root Cause Analysis) initiatives as part of the Strategic Framework • Expanding the use of condition-based/predictive maintenance practices • Development of Asset Condition Monitoring program • Increased monitoring and reporting functionality in CMMS and GeoPortal • Planning and Scheduling • Spare Parts Management • Improvement to asset handoff protocol workflow • Increased involvement of Treatment Plant O&M staff in Capital Project Planning • Using Condition Assessment and Comprehensive Asset Evaluation as tools to guide Capital Project Phasing and Project Prioritization • Staff development • Succession planning • Expanding online training and technology resources for staff 147 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 383 of 461 Page 150 of 195 • Community outreach • Dashboards for key maintenance metrics • Disaster preparedness efforts during Public Safety Power Shutoffs and COVID-19 pandemic • Better access to asset management documents, maintenance history,job reports, and availability of asset spare parts from a web application and work orders • Updating sewer structure asset inventory to capture as-is conditions, photos, structure types, location in a bicycle lane, road type, and flagging structures needing structural changes; as a result, improvements were made to identified structures, and now more information is available to Central San staff • Streamlining work order creation and improving data accuracy with asset barcodes • Providing dynamic dashboards linking data to relational databases that provide near real- time insight into system performance and asset management activities These improvements have been made as a joint effort with engaged stakeholders. Table 21:Asset Management Plan Elements • Plan ElementStatus 1 Asset Management Organization, Policies, and Key Strategies ipm 2 Asset Register AAV= 3 Plant Maintenance Strategy MPM 4 Business Process Mapping MPM 5 Knowledge Management Strategy (On-Going Program-AVM Succession Planning) 6 Condition Assessments (On-Going Program) MVM 7 Business Risk Exposure MVM 8 Failure-Based Maintenance Strategies WPM 9 Assaluations �m 10 Program and Project Management Standards �Wpw 1 1 Asset Commissioning and Handover MM 12 Strategic Forecasting (On-Going Activity) up= 13 Capital Improvement Program Business Case Evaluations MrVW 14 Update Asset Management Plan 148 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 384 of 461 Page 151 of 195 2. ASSET MANAGEMENT PLAN - GOALS FOR THE NEXT 5 YEARS Current related projects underway are described in Table 22. Table 22:Asset Management Projects No. Current Projects Planned Start Duration 1 Migration from Geometric Network to Utility 18 Network 2 Asset Barcode 18 The goals here are based on Uptime Elements and ISO 55000. Many of these goals were initiated by the progress made throughout the years. Table 23:Asset Management Future Goals StartNo. Asset Management Plan Element Planned Duration • 1 Complete the Asset Registry for Treatment Plant Piping 2023 16 Underground Utility 2 Continue Development of Data Analytics 2023 12 3 Continue Comprehensive Asset Evaluation for the 2022 Ongoing Prioritizations of Vertical Assets 4 Digitize Operator Round Sheets 2024 5 Smart Utility Initiatives 2025 36 • Complete Standardization of Smart MCCs across Central San • Pilot Asset Health/Condition Monitoring Technology • Asset Handoff Tracking & Monitoring • Apply/Pilot Artificial Learning/Machine Learning Where Applicable 6 Dynamic Asset Management Plan 2028 12 149 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 385 of 461 Page 152 of 195 3. ELEMENT DESCRIPTIONS A. ELEMENT 1 - COMPLETE THE ASSET REGISTRY FOR TREATMENT PLANT PIPING UNDERGROUND UTILITY This task is to complete the utility locating and surveying/drafting to consolidate treatment plant piping as-builts. Critical data for each treatment plant piping reach are origin and terminus, contents, diameter, material, and installation year. This information will be prepared for import into the current project underway to build upon the Utility Network schema database. B. ELEMENT 2 - CONTINUE THE DEVELOPMENT OF DATA ANALYTICS Staff has been working together across workgroups to develop trending information from bearing temperature and ultrasound inspections. Criteria is being developed for critical temperature zones to understand when unexpected temperature or sound deviations occur. The inspector will also record pre-greased and post-greased Root Mean Square and Crest Factor readings to evaluate the performance of the asset based on the actions taken. This dashboard will serve as a pilot for trending nearly any asset inspected on a preventative or predictive maintenance schedule. The monitoring and reporting methodology allows for greater visibility; "you can't manage what you don't know" leads to future maintenance efficiency improvements. TP-Furnace Blowers 01MI2021DATE:1011412020,12:DO:00 PM WORK OMER ID 498159 i f; , � o, OD �- I ` try+ 1 . Figure 81:Treatment Plant Maintenance Furnace Dashboard Figure 81 and Figure 82 show temperature and sound inspections for the treatment plant furnace. Each data point is a dynamic work order; the user can see the corresponding work order or inspection ID with a CMMS link. 150 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 386 of 461 Page 153 of 195 TP-Furnace Blowers AO TP-Furnace I Blowers Figure 82:Treatment Plant Maintenance Furnace Dashboard of Temperature and Sound Inspections C. ELEMENT 3 - CONTINUE COMPREHENSIVE ASSET EVALUATION FOR THE PRIORITIZATIONS OF VERTICAL ASSETS Using the Comprehensive Asset Evaluation method from the Steam Project, staff will continue comprehensive condition assessment for the vertical assets. The condition assessment methodology involves initial planning and evaluation, inspection, and engineering analysis phases. This methodology can help prioritize the assets for capital improvement at the Treatment Plant. D. ELEMENT 4 - DIGITIZE OPERATOR ROUND SHEETS The Plant Operators perform "rounds" daily to gather data to monitor plant performance. On any given day, they can collect up to 3,300 data points on 18 sheets of paper, which they copy by hand for the Control Room. In a multi-divisional effort, staff from CSO, Plant Operations, Plant Maintenance, and Asset Management have transferred the sheets into Cityworks, and the Operators are currently beta-testing the digitized process on mobile devices. If successful, the project will import and export data easily, and doing rounds will be a more efficient and paperless process. E. ELEMENT 5 - SMART UTILITY/DIGITAL TWIN INITIATIVES To achieve Central San's vison to be an innovative industry leader, staff participated in smart utility vision workshops in 2023 to define a forward-looking destination that the Central San Smart Utility Committee will work towards to ultimately achieve the Central San's vision. The results of the workshops will be forthcoming. ............................................................................................................................................................................................................................................................................................................................................... I. SMART MOTOR CONTROL CENTERS 151 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 387 of 461 Page 154 of 195 As Central San increases its focus on energy optimization, it is installing new smart Monitor Control Centers (MCC) to improve the monitoring of electrical power usage. The data will be available for analysis to optimize processes. Central San will also use the data to troubleshoot equipment problems quickly and bring processes back online faster. The data will help implement changes to reduce energy consumption. ............................................................................................................................................................................................................................................................................................................................................... Il. PILOT ASSET HEALTH/CONDITION MONITORING TECHNOLOGY Central San's Plant Maintenance Division has various asset condition monitoring sensors for asset reliability and optimization. An Asset Health Monitoring tool would provide actionable insights that include asset health scores and status. The goal is to track, update, and analyze assets for risks and RUL. Central San has met with various vendors over the last few months, such as Bentley Systems, Caltrol Emerson Impact Partner, Tango Health, Microsoft Power BI (Business Intelligence), and others, to learn about the different software applications. Central San has not yet chosen a specific software or vendor. These are multiple monitoring sensors at the Treatment Plant. These sensors monitor the Treatment Plant machines such as motors, pumps, compressors, valves, web scrubbers, boilers, or MCC units. The asset health indicator would provide an analysis on: • Bad actors • Condition scores last tested date • Pareto by system, subsystem, or process, and subprocess • Maintenance level • Mean Time to Repairs • Mean Time Between Failures • Cost avoidance • Replacement asset value • Reactive Maintenance and Preventative Maintenance work orders ............................................................................................................................................................................................................................................................................................................................................... III. ARTIFICIAL INTELLIGENCE/MACHINE LEARNING Central San embraces artificial intelligence/machine learning when and where appropriate in alignment with Strategic Plans and the IT Technology Master Plan. Machine learning would leverage data collected as source materials for algorithms to derive information for analysis, prediction, or what-if scenarios. For asset management, these sources of data may include inspections from CCTV, digital rounds, inspections from furnace blowers, or asset maintenance history for predictive analysis. Images, sensor data from physical assets, and SCADA Historian data may also be leveraged for machine learning. Staff will explore these topics as part of the IT Smart Utility Visioning Workshops. F. ELEMENT 6 - DYNAMIC ASSET MANAGEMENT PLAN The information written here is static as soon as it is adopted by the Board until it is updated every five years. In addition to the paper plan, since the collection systems and treatment plant are undergoing capital improvements and evolving rapidly, it is important to provide timely data for making decisions. This element will develop interactive business intelligence tools that will show near real-time information for making decisions. 152 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 388 of 461 Page 155 of 195 Asset Management The coordinated set of activities that an organization uses to realize value from assets in delivery of the organization's vision, mission, and values. The activities involve balancing cost, risk, opportunities, and performance benefits. Asset Condition Information Asset condition information includes data and observations on the condition of an asset. More than just the current state, it is data and information accumulated over the life of an asset. It helps support day-to- day and long-term decisions about asset conditions and future conditions. Asset Condition Management Asset condition management is the overall approach to monitoring, diagnosing, and identifying corrective actions. This can be done by continuous or periodic measurement and interpretation of data to indicate the asset condition (health) and determine its maintenance requirements. Asset condition management includes accumulating and correlating asset condition information and observations over the asset's life. Asset condition management includes predictive maintenance, condition-based maintenance, and other maintenance technologies and techniques. Asset Management Implementation Plan The Asset Management Implementation Plan developed in 2015 by Carollo Engineers and Central San staff advanced the efforts of the Asset Management Program.The plan documented processes and procedures for data management, analysis, and provided six recommended plan elements for implementation. The elements were the following: • Developed organization, policies, and key strategies to support asset management. • Developed asset hierarchy and data standards which resulted in a single registry of assets. • Developed business process mapping to provide a framework for Central San to continue with the development and improvement of business processes for overall efficiency and safe operation. An example developed was the asset handover business process. • Development of knowledge management strategy. • Development of failure modes and condition assessment protocol with condition assessment of Pumping Stations assets and consolidating closed circuit television databases. • Systematic development of a comprehensive maintenance program based on asset criticality and consequence of failure and application to assets. • Identification of the most effective preventative maintenance tasks for treatment plant equipment and systems. • Integration with a computerized maintenance management system to establish a repeatable program with documented processes and procedures. • Implementation of Azteca's Cityworks Asset Management System platform as the computerized maintenance management system for both horizontal and vertical assets; Cityworks replaced two existing platforms, Accela and Mainsaver. • Implementation of the Reliability Centered Maintenance Program. Bad Actors Assets that have high number of failures, high maintenance cost, and unplanned downtime. Condition-Based Maintenance or Predictive Maintenance Condition-based maintenance is non-intrusive task performed to check health of equipment by monitoring or trending parameters that are known to be indicators of asset health. Predictive 153 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 389 of 461 Page 156 of 195 maintenance uses more advanced techniques and technologies that diagnose current state of equipment and aids in predicting when equipment failure might occur. Secondly, predictive maintenance prevents the occurrence of failure by performing corrective maintenance. Monitoring for future failure allows maintenance to be planned before the failure occurs. Ideally, predictive maintenance allows the maintenance frequency to be as low as possible to prevent unplanned reactive maintenance, without incurring costs associated with doing excessive preventive maintenance. Predictive maintenance uses condition-monitoring equipment to evaluate an asset's performance in real-time. Condition-based maintenance and predictive maintenance are used to predict failures before they occur to keep unplanned maintenance to a minimum because unplanned maintenance costs 8 to 10 times more than planned maintenance. Equipment with rotating elements is a natural fit for vibration analysis, electrical equipment for thermography, and so forth. In some cases, the devices monitoring the asset may be connected to a building automation system, a distributed control system, or a programmable logic controllers system. Predictive maintenance includes trend analysis, pattern recognition, and other correlation analysis. Condition-based maintenance and predictive maintenance can be online continuously or based on periodic inspection. Condition-based maintenance optimizes preventative and corrective actions by avoiding traditional run time or calendar-based maintenance tasks. At Central San, staff uses condition-based maintenance and predictive maintenance interchangeably in the computerized maintenance management system. Corrective Maintenance Corrective maintenance is a planned activity that may be an activity stemming from a reactive maintenance, preventative maintenance, or predictive maintenance and is any maintenance activity required to correct a failure that has occurred or is in the process of occurring to ensure the asset or equipment is restored to working condition, examples include repairs or replacement. Corrective maintenance may also include improvements to over-system or subsystem robustness. Corrective maintenance can be subdivided into immediate or deferred corrective maintenance (the latter is delayed based on a given maintenance or capital improvement rules). The Plant Maintenance Division performs corrective maintenance on assets. The work is usually identified by Plant Operations staff or when Plant Maintenance staff performs preventative or predictive maintenance on the asset. Defect Elimination also known as Don't Just Fix It Improve It Defect elimination is a reliability strategy to eliminate known defects caused by aging,wear and tear, workmanship, changed operating conditions requiring more robust components, or inadequate replacement parts that don't meet current stress levels present in an asset. Defect elimination analysis meetings typically can be completed in a day because they deal with known defects. The total cost is less than $10,000 and the work does not include replacement of an asset that costs$5,000 or more with a useful life of more than one year. Above these threshold values, the work would be part of a capital project. Defect elimination is typically a follow up action(s) from a preventative maintenance, predictive maintenance, or corrective maintenance task. Failure Modes and Effects Analysis Failure modes and effects analysis is a methodology to examine an asset, process, or design to determine potential ways it can fail and its potential effects on its function. Failure effects are examined locally at the equipment, at the process, and at the plant level. It identifies appropriate mitigation tasks to prevent functional failure. 154 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 390 of 461 Page 157 of 195 Failure modes and effects analysis is an ideal tool in the design phase of the asset lifecycle where operational data is scant and/or unavailable. Engineers prioritize the failure modes by scoring three separate factors on individual scales of 1 to 10: likelihood of occurrence, severity of failure, and effectiveness of control systems, such as performance-monitoring systems. Failure Reporting, Analysis, and Corrective Action System Failure reporting, analysis, and corrective action system is a process by which failures can be reported in a timely manner, analyzed, and a corrective action system put into place in order to eliminate or mitigate the recurrence of the failure. Monitoring is based on failure, cause, and action codes. Analysis can include reliability centered maintenance, planned maintenance optimization, root cause analysis, defect elimination, or safety, quality, and effectiveness type of follow up. Maintenance Activities to keep assets and components in proper operating condition as designed to achieve its intended function and desired service levels.To keep in existing state, preserve, continue in good operating condition, and protect. Maintenance Improvement/Optimization/Modification Work Maintenance improvement is an activity directed at equipment, asset, or system related improvements or modifications, typically come from continuous improvement ideas or a reliability analysis. The total cost is less than $10,000 and work does not include replacement of an asset that costs $5,000 or more with a useful life of more than one year. Above these threshold values, the work would be part of a capital project. It can be broken down further into corrective maintenance - maintenance improvement. Maintenance Management The management and control of the maintenance function. The coordinated set of activities performed on the assets. Maintenance, Repair, and Operations Maintenance, repair, and operations describes the actions needed for a utility's upkeep for production, the systems within the utility, and the assets needed for operating that utility. These actions include running equipment repair and maintenance, material handling equipment repair and maintenance, supply chain management, tools and consumables for maintenance and repairs, and infrastructure maintenance. Non-Maintenance Work Non-maintenance work is typically, building related non-maintenance type activities such as: office moves, Multi-Purpose Room set up, meetings, trainings, or special projects. A special project is work activity directly related to a specific pilot project or a project. This includes equipment, labor, and materials (special project). Special project work should be captured as Capital Improvement under expense type on the work order. Planner Update Planner update is a process that allows any Technician to make suggested changes to an asset record or work order task and notify the Maintenance Planners. By checking a box on the work order, the suggested changes are automatically communicated to the Maintenance Planners and tracked in the computerized maintenance management system. Typical changes include updating asset details such as an attribute field, modifying a procedure in the preventative maintenance, or adjusting the frequency of the task. Also, correcting wrongly assigned assets to a work order or duplicate work orders. The 155 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 391 of 461 Page 158 of 195 procedure was established to provide a quality assurance/quality control and continuous improvement feedback loop between the Technicians, Supervisors, Maintenance Planners, and Managers. Preventive Maintenance A preventive maintenance is maintenance that is regularly performed on a piece of equipment to lessen the likelihood of functional failure. It is performed while the equipment is still working so that it does not break down unexpectedly. In terms of the complexity of this maintenance strategy, it falls between reactive maintenance and predictive maintenance. Preventive maintenance action is conducted to keep equipment or assets working and extend the life of the equipment or asset. Preventive maintenance may include tests, measurements, adjustments, cleaning, and parts replacement, performed specifically to prevent faults. • Types of preventive maintenance: o Preventive maintenance can be scheduled on a time or usage based trigger: ■ Time-based preventive maintenance: • A typical example of a time-based preventive maintenance trigger is a regular calendar-based inspection on a critical piece of equipment that would severely impact production in the event of a breakdown. • Usage-based preventive maintenance: • Usage-based preventive maintenance triggers after a certain number of miles, hours, or production cycles. An example of this trigger is a cake pump which might be scheduled for service every 3,000 hours. ■ Failure finding: • A scheduled task to see if a hidden failure has happened or is about to happen. Planned Maintenance Optimization Planned maintenance optimization is a methodology that focuses on improving maintenance effectiveness and efficiency by reviewing an existing maintenance program and, in most cases, adding maintenance tasks to account for failure modes not addressed by the existing program.Typically applied to better-to-well-behaved systems (20/80 systems -where only about 20 percent of all problems develop) are candidates for experienced centered maintenance analysis. Experienced centered maintenance was adopted for the 20/80 systems to minimize impact of Treatment Plant and Pumping Stations personnel engaged in the overall reliability improvement initiative and reduce financial impact on the Treatment Plant budget for the improvement initiative. Each system requires a "quick look" (two to four days) by an in-house multi-discipline analysis team led by a specialist. Reactive Maintenance Reactive maintenance (also known as breakdown maintenance or emergency maintenance) refers to repairs that are done when equipment has already broken down, in order to restore the equipment to its normal operating condition. Maintenance occurs when someone interacts with or recognizes a problem with an asset or with an infrastructure system. It may also have been called "unscheduled work" or "schedule busters". Typically, we did not see the failure coming. No preventative strategy, strategy did not work, or preventative maintenance was not performed. Reliability The probability that an asset will perform its intended function satisfactory under specified conditions within a specified period. 156 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 392 of 461 Page 159 of 195 Reliability Analysis Reliability analysis is the process of identifying the maintenance needs of significant items and identifying the impact of malfunction or failure to safety, environment, operations, and financial. The possible failure modes of the items are identified, and the appropriate maintenance policy and tasks are assigned to mitigate the impact. Reliability analysis includes reliability centered maintenance, planned maintenance optimization, defect elimination, and root cause analysis. Reliability and Maintenance Framework A reliability and maintenance framework is a road map of how to apply the reliability and maintenance requirements to optimize a maintenance program. Central San currently utilize Pareto analysis based on count and cost of reactive work orders to identify process, subprocess, and assets subject to program review. Other methods of identification include bad actors and internal reliability analysis tools for gap analysis. Reliability Centered Maintenance Reliability centered maintenance is a systematic approach to developing a comprehensive maintenance program based on asset criticality and consequence of failure.This method utilizes qualitative decision methodology that identifies the most effective preventative maintenance task for plant equipment and systems. Because of the analysis and software, it establishes a repeatable program with documented processes and procedures. Reliability Strategy A systematic approach to development of the reliability and maintenance requirements to optimize a maintenance program. Rework A repeat failure of the same part and same failure mode within 90 days of any preventative or corrective maintenance completion date. Risk Threshold Investigation Risk threshold investigation is used for 20/80 systems that may never be subject to reliability centered maintenance and an even shorter (one day or less) alternative analysis for 20/80 well-behaved systems. The basic goal of risk threshold investigation is to qualitatively identify potential problems or failures in 20/80 (well-behaved) systems that could: 1. Realistically occur and produce or affect one or more of the seven consequence areas (safety, environment, downtime, operations, regulatory requirement, single point failure, and economics) and 2. Cross a threshold that is judged to be sufficiently severe to warrant special attention to mitigate or eliminate risk. This may lead to further reliability analysis. The risk threshold investigation process utilizes the experience and judgment of a team of experienced technicians and/or first line maintenance supervisors, and an operator does the analysis. Analysis is facilitated by a specialist in risk threshold investigation methodology conducting the five-step process. These steps mimic, to a certain degree, the analysis employed in the classical reliability centered maintenance studies but stop far short of the comprehensive information collected and deliberation conducted in classical reliability centered maintenance. The whole risk threshold investigation process typically takes less than one workday. 157 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 393 of 461 Page 160 of 195 Root Cause Analysis Root cause analysis is a process that is applied when situation or problems occur that demand attention to mitigate or eliminate them and there is uncertainty about all the contributing causes. Root cause analysis is needed because reliability centered maintenance and planned maintenance optimization or experienced centered maintenance can't be expected to identify all the contributing and/or latent factors that result in failures occurring in a system lifecycle, especially those involving humans. Poorly documented maintenance processes, procedures, and work instructions can all be contributing causes to failures even after the most thorough reliability centered maintenance or experienced centered maintenance analyses are completed. New personnel, lacking proper training and experience and rushed into a crisis, can overlook critical steps in completing repairs, resulting in infant equipment failures shortly after resumption of operations. The changes in replacement parts, lubricants, and other items from the organization supply chain can be causes contributing to a failure. The changes in operating conditions, system configuration, or practices can also be contributors. Root cause analysis investigations may take several days and the report of findings and recommendations for corrective action can be lengthy. Run to Failure Run to failure is a maintenance strategy for assets where the cost and impact of failure is less than the cost of preventative actions. It is a deliberate decision based on operational, economical, safety, and environmental factors, not to perform preventative maintenance, but replace or rebuild asset upon failure. This means spare parts replacement assets are readily available and easily restored to service without impact to operations, safety, environment, or significant economic cost. Safety, Quality, and Effectiveness Observation Safety, quality, and effectiveness is an internal process to routinely review the preventative maintenance program based on observation of the preventative maintenance or predictive maintenance tasks. 158 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 394 of 461 Page 161 of 195 Agency, U. S. (n.d.). About Asset Management. Retrieved from Asset Management: A Best Practices Guide: https://nepis.epa.gov/Exe/ZyPdf.cgi?Dockey=P1000LPO.txt Barnett, E. (2020). Crisis Communications Plan. Matinez: Central San. California State Water Resources Control Board. (2023, February 10). Sanitary Sewer Systems General Order. Retrieved from Sanitary Sewer Systems General Order: https://www.waterboards.ca.gov/board_decisions/adopted_orders/water_quality/2022/wq 0_2022-0103-dwq.pdf Carollo Engineers. (2015). Asset Management Implementation Plan Summary Report. Walnut Creek: Carollo Engineers. Carollo Engineers. (2017). Comprehensive Wastewater Master Plan. Walnut Creek: Carollo Engineers. Central San. (1987). Engineering Department Project Procedures Manual. Martinez: CCCSD. Central San. (2006). Recycled Water Program Manual. Martinez: Central San. Central San. (2012). Central Contra Costa Sanitary District Strategic Plan FY 2022-2024. Martinez: Central San. Central San. (2016). Resolution 2016-046, A resolution of the Central Contra Costa Sanitary District setting forth and delegating certain authorities of the General Manger and rescinding Resolutions Nos. 2015-010 and 2016-018. Martinez: Central San. Central San. (2019). Operating Modes. Retrieved from Plant Information Management System (PIMS): https://pims.sancentra1.org/pod/operating-modes. Central San. (2021). Annual Comprehensive Financial Report. Martinez: Central San. Central San. (2022). Central Contra Costa Sanitary District FY 2022-23 Budget. Martinez: Central San. Central San. (2022). Central Contra Costa Sanitary District's 2022-2024 Strategic Plan. Martinez: Central San. Central San. (2022). Sewer System Master Plan. Martinez: Central San. Central San. (2022). Standard Specifications for Design and Construction. Martinez: Central San. Central San. (Fall 2022). 25 Years of Household Hazardous Waste Collection. Pipeline, 8. Cortinovis, D. (2011). Emergency Response Plans Wastewater Pumping Stations. Dan Cortinovis. Deutsch, S. (2016). Emergency Operations Plan. Martinez: Central San. Deutsch, S. (2022). Emergency Operations Plan. Martinez: Central San. 159 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 395 of 461 Page 162 of 195 EPA, U. (2021). Asset Management for Water and Wastewater Utilities. Retrieved from https://www.epa.gov/sustainable-water-infrastructure/asset-management-water-and- wastewater-util ities HDR. (2021). TM No. 20 Hydraulic Capacity Evaluation: Aeration Basins Through Effluent. Walnut Creek: HDR. HDR. (2022). Wastewater Characterization Results. Walnut Creek: HDR. Lawson, D. (2016, November 7). Board Policy 015, Asset Management. Matinez. Lawson, D. (2017). Cityworks Custom Business Process Manual. Martinez: Central San. Lawson, D. (2018). Asset Mangement Procedures Manual. Martinez: Central San. Ledbetter, J. (n.d.). Emergency Action Plan. Martinez: Central San. Raftelis Financial Consultants, Inc. (2015). Wastewater Cost of Service Study. Pasadena: Raftelis. RMC Water and Environment. (2015). Wholesale Recycled Water Opportunities. Walnut Creek: RMC Water and Environment. West Yost. (2019). Recycled Water Model Update and Hydraulic Analysis. Walnut Creek: West Yost. 160 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 396 of 461 Page 163 of 195 1 . BOARD POLICY 015, ASSET MANAGEMENT Number: 1 015 Authority. Board of Directors _ Effective: November20,2014 Revised: November 3,2016 Reviewed.July 21,2020, August 2,2022 Initiating DeptdDiv.: Engineering)Planning&Development BOARD POLICY ASSET MANAGEMENT PURPOSE The purpose of this Board Policy is to recognize that the District ■ has invested significant resources in its assets,and • aims to optimize the management of these assets to deliver high quality and reliable services in a sustainable manner with an acceptable level of risk. DEFINITION An asset pursuant to this Policy refers to both physical and software assets and is defined as having a replacement value greater than$5,000 and a useful life of at least two years. POLICY Asset management is a broad strategic framework that encompasses many disciplines involving the entire District.In support of its Mission,Vision,and Strategic Plan,the District sets forth the following directives related to asset management: Data and Knowledge The District will maintain a comprehensive asset register to facilitate management over the assets'lifecycle. Information Systems The District will make capital and ongoing investments sufficient to own,operate,and maintain a reliable and state-of-the-art security-protected information management system that supports timely,evidence-based decision making at all levels of the organization. Processes and Practices The District will develop processes,procedures and practices for staff in support of this Policy using industry accepted guidelines and practices to provide consistency across the organization. People The District will implement education,training,and mentoring programs to ensure that asset management practices are widely communicated,understood,and implemented throughout the organization. Organization The District will establish organizational responsibility to lead and manage a comprehensive asset management program. [Ongrnar Retained by the Secretary of the District] 161 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 397 of 461 Page 164 of 195 2. WHAT IS ASSET MANAGEMENT? �rEPA Asset Management: A Best Practices Guide United States Environmental Protection Agency Introduction This guide will help you understand: • What asset management means. PA"e • The benefits of asset management. • Best practices in asset management. • How to implement an asset management program. - T"Z,t This guide is intended for owners, managers,and operators of water systems, Audaeme local officials, technical assistance providers, and state personnel. Asset Nlanagement Asset management is maintaining a desired level of service for what you want your assets to provide at the lowest life cycle cost.Lowest life cycle cost refers to the best appropriate cost for rehabilitating,repairing or replacing an asset.Asset management is implemented through an asset management program and typically includes a written asset management plan. Challenges faced by Benefits ofAsset Nlanagement Water Systems • Determining the best(or optimal)time • Prolonging asset life and aiding in jto rehabilitate/repair/replace aging rehabilitate/repair replacement assets. decisions through efficient and focused operations and maintenance. • Increasing demand for services. 1 • Meeting consumer demands with a y' • Overcoming resistance to rate focus on system sustainability. increases. • Setting rates based on sound ,- • Diminishing resources. operational and financial planning. • Rising service expectations of • Budgeting focused on activities critical customers. to sustained performance. • Increasingly stringent regulatory • Meeting service expectations and requirements. regulatory requirements. • Responding to emergencies as a result • Improving response to emergencies. of asset failures. • Protecting assets. • Improving security and safety of assets. Implementing Asset Nlanagement:Five Core Questions Framework A good starting point for any size water system is the five core questions framework for asset management. This framework walks you through all of the major activities associated with asset management and can be implemented at the level of sophistication reasonable for a given system. These five core framework questions provide the foundation for many asset management best -, practices. Several asset management best practices are listed for each core question on the following pages.Keep in mind that these best practices are constantly being improved upon. 162 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 398 of 461 Page 165 of 195 3. SERVICE AREA AND ASSETS m - 6."aieviYe�w Concord Clyde 1.d.6trlal lt6y .••' �,',I,a r' t i 11 e Z Headquarters,Treatment PIaat �] i a r d ad FIRW Collection Facility Buchanan Field North Buchanan Field South O xt c P l e a s n n t C o iti [ I C l a y t o n Sleepy Hollow µ a E n u t L a i�a y c I l c A cae'a Collection System 1 1 - 0 RpeN io b Flush lSileen,Y �� j >�) Low i otinda\ ( HBa[es Blvd Oriada . . "1C+oge[oads O r i tt d a MOraga� IVI o C a 4� k 11 y a l r S d 11 R. d. in O it Administration/Operations Facility '. all Lift Station ° S/ .� Pump Station Private Pump Station — Recycled Water Main f Sewer Force Main(?4°) San Ramon costa"'w Sewer Main(<12") Trunk Sewer Main(>12'<24") -- \ coP10medd — Interceptor Sewer Main (�24") €. Outfall Recycled Water Distribution Zone 1 Service Area 0 Wastewater Collection and Treatment Service Area r— Household Hazardous Waste Collection Service Area Wastewater Treatment and Household Hazardous Waste Collection Service Area 163 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 399 of 461 Page 166 of 195 The Sewer System Management Plan, along with the Sanitary Sewer Overflow & Backup Response Plan, Sewer System Management Plan Position Paper, Central San's Code of Laws, Standard Specifications, and the Sanitary Sewer Overflows Annual Report can be found at Central San's website: https://www.centralsan.orq/post/sewer-system-management-r)lan System Statistics can be found here on these dashboards: Sewer: https://aeoportal.centralsan.ora/portal/apps/dashboards/a5a865dc277e4d58a40c46daf033d2b9 Treatment Plant: https://aeoportal.centralsan.org/portal/apes/dashboards/d966729cf31 c46a39afbd7359ede717e Service Area Infrastructure Estimates Asset Total Pipelines, estimated miles 1,538 Manholes,each 31,564 Pumping Stations,each* 18 * 3 are private but maintained by Central San 4. ASSET MANAGEMENT ORGANIZATION CHART Central San Management Team and Organization Chart can be found here: https://www.centralsan.org/node/803 164 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 400 of 461 Page 167 of 195 5. RECYCLED WATER DISTRIBUTION Central San Recycled Water System Map Date:4/13/2022 � O Martinez r , /�� J v �l DO �oo�aO�O Recycled Water Pipeline CCWD Pipeline Area Concord �D p Detail 1 O .� osed Extension , � ® � Pro Pleasant Hill Martinez Water Service Area d 10, Recycled Water Zone 1 �fayelle Walnut creel( PbP= ' Recycled Water Customers L i , "` \�O ° (size indicates yearly consumption) may` s Small M Medium L Large DMA 1� �" CITY OF CONCORD ® ® ® Existing Customer MEDIANS " DEUTSCHER PROPERTIES- - 1 _ BUFFALO WILD WINGS SEAFOOD CITY ■ .- WI PARKLLOW as ■ BUSINESS PARK ` HILTON e , AUTO-FILL STATION !� HOTEL AGLANTIS ' i `1100 CONCORD AVE SECURITY OWNER CORPS VERANDA ai�yfo ee ! ��po r .r�r;. PFU 1 TRUCK FILL COUNTY ■� BELLECCI&ASSOCIATES I QUARRY HYDRANT BUCHANAN FIELDS GOLDEN STATE GOLF COURSE UMBER IN PLANT USE �//CA TEACHERS PLANT HYUNDAI ASSOCIATION P.H.CORP.YARD TAYLOR BLVD/RUTH I PLEASANT HILL `OPERATIONS CARMAX DRIVE LANDSCAPING - MIDDLE SCHOOL BUILDING 3 TRUCK FILL FORD CHILPANCINGO SEQUOIA HYDRANT PARK © MIDDLE DEMOPLEASANTHILLGARDEN \, o- P.H.PARK ��V1m'11I SCHOOL DEMO GARDEN -" HEADQ BUILDINGOFFICE���� VIKING DR E� F.SALFINGERE PARK CONTRA COSTA ® DVC P.H.OL_ICE DEPT CC TOPSOIL © HOB ANNEX -�'�` BLVD MEDIANS CAMPUS ` SEQUOIA �! �S yI-- CHILPANCINGO J ®�_ ELEMENTARY COUNTY CORP. CCCSD 99 BLVD MEDIANS ® TAYLORlCIVIC SCHOOL YARD HHWCF WAREHOUSE ■ STU13BS RD MEDIANS PLEASANT OAKS CONCO ® ` DIABLO VALLEY EY PLAZA © TRIPLEXES _iK° ` PARK COUNTY ANIMAL GOLF CLUB/OLD MONTICELLO AVE SERVICES �■� QUARRY,ROAD MEDIANS '� I` VALLEY VIEW& ' LANDSCAPING ® ® ■ �� l - COLLEGE PARK SCHOOL{S SEASON'S ■ ■ MALTBY ■� ■ �,-': 1 �d CEMETERY ■ �■ /// ` PUMP STATION 9� ■ ■ � m r _ ■ -'l���i���r�� anal c.° ■K ■ "C [ P y9C a¢ ■r ■ ` \Pac _ _ +�� CCWD Q ■a w+ c� _ Gta - ■ Q+ CONTRA COSTA MUD �c • + ` COUNTRY CLUB MARTINEZ ■ ���ItiLF COURSE '•1y 13u11 • ' ■ —0 Ilk _ Feet 165 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 401 of 461 Page 168 of 195 6. CITYWORKS CUSTOM BUSINESS PROCESS MANUAL The cover page and table of contents are provided below. The full version is available through Laserfiche. I Central Contra Costa Sanitary District Updated:April T,201T Standard Operating Procedures Author Dana Lawson Cityworks Custom Business Process Manual Overview Purpose This document covers the custom business processes for Citpurks AMS for Central Contra Costa Sanitary District(Central San). It is intended to complement training and be as a reference_ Revision pate Author Version Change Reference History 10119116 D.Lawson 1.0 First draft 04107117 D_Lawson 1.2 Removed Plant Maint section which will be incorporated into their Maintenance Management Manual References +Cityworks 2015 Training Operations Manual(Woolpert,2016) •Configuration Guide(Woolpert,2016) *CMMS Replacement,District Project 8242 Training Training was provided by Woolpert to system Admins,Stewards,and Supervisors in September 2016(CSO)and January 2017(Plant Maintenance). Training was provided to craftttechs by Asset ManagernenVGIS staff in 5eptemberlOctober 2016(CSO)and January 2017(Plant Maintenance). Mobile training was also provided to CSO in December 2016 and will be Plant Maintenace in JanuarylFebruary 2017 when tablets are ready for deployment by IT. Train-the-Trainer was agreed upon by Stakeholders during implementation with the consultant,which means that training of techsirraft shall be provided by the Supervisors and Stewards for each Division_ Any additional training is the responsibility of the workgroups Supervisors and Stewards. N:1EnwSRVLAM-G1S18242-CNINIS ReplacemerrN58 fl&M ManuarsWDS-CMMS-0134-Ciryworks Custom Business Process Manua1AkCX Page 1 of 116 166 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 402 of 461 Page 169 of 195 Central Contra Costa Sanitary District Updated:April 7,2317 Standard Operating Procedures Author.Dana Lawson Table of Contents Overview........................................................................................................................ 1 Wc"ow Process......................................................................................................................4 Global Activity Statuses..................................................................................................5 Global Workflow(Business Process)Saernatics---------------------------------------------------------8 Roles&Responsibilities---------------------------------------------------------------------------------------------------------22 Admins--------------------------------------------------------------------------------------------------------------------------------------24 Overview----------------------------------------------------------------------------------------------------------------24 (1)Asset Sat-Up and Modification-------------------------------------------------------------------------------25 (2)Employee Set-Up and Modi=5catiori---------------------------------------------------------------------------27 (3)Updating Labor and Cyer­)ead Rates Annually........................................................33 (4)Holiday Set-Up and Modification..............................................................................35 (5)Report Set-Up and Modification---------------------------------------------------------------------------36 Stewards------------------------------------------------------------------------------------------------------------------------------37 Overview-----------------------------------------------------------------------------------------------------------------------37 (6)Preventive Maintenance Set-Up and Modification....................................................38 {7)Grew Set-Up and Modification(Domain Viewable)..................................................A3 (8)Attachment Set-Up and Modification for Templates.................................................A5 (9)Project Set-Up and Modification..............................................................................A7 (10)Employee Skill Set-Up and Mod cation-----------------------------------------------------------------49 (11)Equipment Set-Up and Modification---------------------------------------------------------------------51 (12)Contractor Set-Up and Modification.........................--------------------------------------------52 (13)Material Set-Up and Modification............................................................................53 Supervisors--------------------------------------------------------------------------------------------------------------------------------54 Overview--------------------------------------------------------------------------------------------------------------------54 (14)Initiating-------------------------------------------------------------------------------------------------------------55 {15)Emergency(bypasses rKwmal procedures).............--------------------------------------------59 (16)Screening...............................................................................................................62 (17)Planning.................................................................................................................67 (18)Scheduling.............................................................................................................72 (19)Assigning-------------------------------------------------------------------------------------------------------------74 (20)Re-Assigning Work------------------------------------------------------------------------------------------------78 (21)QA,Reviewing,and Closirmj...................................................................................80 Craft--------------------------------------------------------------------------------------------------------------------------------------82 Overview.......................................................................................................................82 (22)Inspecting the Vehide or Equipment......................................................................83 (23)Performing-------------------------------------------------------------------------------------------------------------85 (24)Completing IQA Retuned----------------------------------------------------------------------------------89 CollectionSystem.....................................................................................................................go Overview.......................................................................................................................go (25)CMMS Section.......................................................................................................91 (26)Paving Work Orders...............................................................................................92 (27)Overflow Response................................................................................................94 14:Ew.rr,9RVLAM-G1S0242-CIVINS RepiacemenMB 09M ManuaMPIDS-GIVINIS-01-34clywrks CuslDm Business Process MBnual.doex Page 2 of 116 167 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 403 of 461 Page 170 of 195 A!rlf"RL Central Contra Costa Sanitary District Updated:April 77 2017 Standard Operating Procedures Author: Dana Lawson (28)Fleet Servioes........................................................................................................96 Treatment Plant,Pumping Stations,General Facili-tioes..............................................................97 Overview......................................................................................................................97 (29)Plant Operations Division--------------------------------------------------------------------------------------go (30)Process Cord"Control Systems Section...........................................................102 (31)Safely Sectian.......................................................................................................103 GeneralStaff--------------------------------------------------------------------------------------------------------------------------104 Overview.....................................................................................................................104 (32)Request for Buildings 9 Grounds Shop................................................................105 (33)Request for CCTV Section...................................................................................107 (34)Risk Management------------------------------------------------------------------------------------------------log Cftywo*s&CMIVIS Training Program-------------------------------------------------------------------------------------110 Overview.....................................................................................................................110 Agenda:Asset Registry&Plant Maintenance Program...............................................111 Agenda:Cityworks.Basic Training...............................................................................112 Agenda:Citymrks.Training for Techs!Craft----------------------------------------------------------------113 Agenda:G"rks;Mobile Training for TechsCraft-----------------------------------------------------114 Agenda:C"rks Traini rig for Supervisors................................................................115 Agenda:C"rks Basic Training for General Staff....................................................116 14:EnvrSRV-AM-GlMG242-CMPAS Replace menffgC&M ManualsAPDS-CMMS-01-34-Cityw rksCuskmn Business Process Manual.ducx Page 3uf 110 168 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 404 of 461 Page 171 of 195 7. ENGINEERING DEPARTMENT PROJECTS PROCEDURES MANUAL Central Contra Costa Sanitary District PROJECT PROCEDURES MANUAL ENGINEERING DEPARTMENT TABU OF CONTENTS No. Sig. Title Last Initial Effective Date Revision Dada Administrative — A-1 DSG Charges to Prvicedures 0511312014 12/13 005 A-2 AJA Position Papers 0511312014 0111611987 A-3 DSG Apqmval of Train� 0511312014 0311811987 A-4 AJA Procesvng Insurance Cerhkates 05//312014 /01011/988 Planning Phase P-1 DSG District Project and Jab Numbem 051131P014 0711 Z2007 P-2 AJA Mannirig for Calleclion System Sewer Perrovafran 05/132014 Unknown falqacfs PT DSG e,nerffs with Outside Agencvs or Carripenws 05/1312014 07/1 1988 P-4 DSG Armaxafran Of LPropedies to the Disbct OM312014 031i911987 Design Phase D-1 AJA Dasw Re~!2�vl Projects _ 0511312014 1"111980 D-2 AJA Daso?for Collection System Sewer Reriovetion 0511312014 021191 14 D-3 DW Signature 8kcks an Final Plans and ac►fkaWns I W1=14 02/01/1987 _ M Phaa4 84 AJA 8W Procsdure 0511312014 0917512005 8-2 AJA AWWOWofBlds 0511312014 091152005 8-3 AJA ProW Canfmanca 051132014 09/11VX06 84 AIJA Responding to Bidders"Raquesfs for ClarallicaabDn, OW1312014 09115J2005 Preparation and Issuance of Addrenda, and incorporaMn of Addenda Into Bid r_iocurnants Consbwchaa Phase C-2 AJA Pmconsfmdion 0'1?312Q14 IVI&'6�05 G-3 AJA Notice to Proceed-Conshuchan Con&wfs 0"312074 C-4 AJA Consultant P=trraA7 F Policy and Amcedure 03113r2014 OW112006 Financial F1 DSG CappWal#Mmvement Budget A#ocattian(CIBA) 0511312014 1=22007 Reqfuests F-2 DSG Use of l-ITF Ccrmcfion NoOftation Roquest 0511YR14 OAr i/19f31 F-3 DSG Subac unt Sud f Change Farm SCF 05.1131P014 p3+l illg$,5 F-4 AJA Approving Wvoices for Payment of Profssslcrral 0511312014 OfA7W991 and Technical Services Agreements Legal Guidelines for Pre-trhigation&Ufigafion 051431- 7/27/Mf Documents tr-2 DSG Recording Documents at the Recorder's 01 ce OtjOYM14 AO(Y2 7 and Picking up Assessof's Maps N:WES JPtPmcsdums%S-indard¢ee Procedures Eng DsprU%Tsble of CorMnts.tlxx 169 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 405 of 461 Page 172 of 195 8. COLLECTION SYSTEM ASSET HANDOVER WORKFLOWS COLLECTION SYSTEM CAPITAL PROJECTS COLLECTION SYSTEM DEVELOPER JOB POST-CONSTRUCTION PROCESS PRE-CONSTRUCTION PROCESS Capital Projects ' inspectsnotifies Asset CP notifies AM of CCTV crew Management (AM) of changes to design during and verifies acceptable changes to design during construction, which is construction quality construction, which is updated in Central Portal updated in Central Portal Mainline AM checks spatial Review sends AM accuracy and AutoCAD completeness of Survey data is used Engineering design file and submittal to update Assistant enters PDF design and approves or attributes and new features as drawings for requests refine existing map "Tentative"status AM updates • date corrections and features CP notifies AM that and status of tentative When CCTV work order is Upon project completion, review resubmittal statusrecord drawings are main and structures. If closed, AM is notified via complete and uploaded applicable, existing main Cityworks to Laserfiche • • - • of date is updated AM uses • • drawings to update all assets and verify •• • is correct 170 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 406 of 461 Page of • COLLECTION DEVELOPER • POST-CONSTRUCTION 1 • • l1 • PROCESS 1l1 • Construction Inspectors CCTV crew TV's line and When CCTV work order is 171 Central Son create CCTV New Line verifies acceptable closed, AM is notified via Acceptance Work Order construction quality Cityworks in Cityworks AM updates project Upon project AM updates install date tracking information and completion, Mainline and status of tentative sends redlines and clean Review sends AM main and structures. If PDF to CAD group to Construction Plan applicable, existing main create record drawings Redlines status and end of life date is updated AM uses Record CAD group notifies AM Drawings and redlines to that Record Drawings update any additional are complete and affected mains and loaded to Laserfiche structures and check for data accuracy Asset . ag - • Page i of • TREATMENT PLANT CAPITAL PROJECTS DESIGN PHASE AM Extracts Asset Info AM provides access to CP coordinates with exported spreadsheet design consultant to Capital Projects (CP) from Asset Registry into to design consultants, update Asst Handover requests asset list Asset Handover Spreadsheet Maintenance Planners, spreadsheet based on and CP design Maintenance and AM Reviews and CP Distribute Design Warehouse coordinate provides comments on Maintenance assigns Submittals for spare parts updated information new Asset Position Maintenance and information and as-needed support Numbers requests new Asset Position Numbers AM receives Asset Engineering Assistant Engineering Assistant Handover Sheet with adds new assets to enters new Asset ID Asset Numbers for new database in number into Asset assets and a list of "Reserved" status Handover spreadsheet those being replaced for reference purposes Asset172 Central Son . ag - • Page of • TREATMENT PLANT CAPITAL PROJECTS CONSTRUCTION AND CLOSEOUT PHASES AM Extracts Asset Info AM provides access to CP coordinates with from Asset Registry into exported spreadsheet design consultant to CP requests asset list Asset Handover to design consultants, upd andover ate Asst H Spreadsheet Maintenance Planners, spreadsheet based on and CP design Maintenance and AM Reviews and CP Distribute Design Warehouse coordinate provides comments on Maintenance assigns Submittals for spare parts updated information new Asset Position Maintenance and information and as-needed Numbers requests new Asset support Position Numbers AM receives Asset Engineering Assistant Engineering Assistant AM uses record Handover Sheet with adds new assets to enters new Asset ID drawings to update all 173 Central Son Asset Numbers for new database in number into Asset assets and verify data assets and a list of "Reserved" status Handover spreadsheet is correct those being replaced for reference purposes Asset . ag - • Page 176 of 195 9. TREATMENT PLANT ASSET HANDOVER WORKFLOW I • •• • .• f•- • 8/30/2023 �n Finalize warehouse stock a w Accounting Discuss changes in stock Review spare parts changes Accounting ' Add/Remove Request to stock(min/max) Notify Ops/Maint when transmits Close capitalizes/ . a7 depreciates assets in W 3 new items are added Out Form ERP NG manages design phase, As needed,ENG creates ENG manages: ENG receives,reviews, ENG requests ENG releases ENG completes ENG manages: and accepts record acceptance of retention after and returns routes design submittals, WO in Cityworks for acceptance testing construction submittals drawings/documents contract work(GC- required time form to and incorporates comments asset install assistance from PLANT start-up from Contractor 41) (GC-41) accounting (Erg Proc D-1) MAINT training (Section 01620) 4, ENG updates asset t 1 1 spreadsheet w/ ENG uploads 1 1 warranty dates and electronic asset ENG creates master asset I I spreadsheet,using template i 1 sends to PLANT reference/O&M L ENG requests asset ENG submits: I MAINT Manualto W lists foralle ui equipment provided by AM/GIS,of new, r--- q P I PM/SOP's and I Laserfiche replaced,and disposed c in the Process/ frequency from 1 ENG distributes ticSubprocess S stem assets and sends to PLANT /y I manufacturer 1 asset reference 1++ the j roect affects for MAINT,GIS,OPS,and P j I manuals(Section ++ Regulatory Compliance with u planning purposes I I 01620)to OPS& � design submittals(typically I I O 1 I PLANT MAINT at 50%and 95%).If list is I 1 CAD reviews, a created b a consultant,En I I ENG reviews Y g will QA/QC before passing j ENG transmits CAD version of finalizes,stores,and CAD reviews distributes the along 1 I ecord drawings to drawings and I 1 record drawing record drawings and CAD staff(Section resubmits load to laserfiche 01640) submittal 1 I I 1 I I 1 I I 1 1 I I I 1 I 1 PLANT MAINT completes the W0;if j PLANT MAINT reviews fina I 0&M 1 I needed,Maint Planner will create a 1 Manual and spreadsheet.Then PLANT MAINT reviews design 1 1 submittals: 1 `Project"to group the WOs 1 sends asset registry updates to GIS: asset/position number/name W a maintenance/access 1 1 location PLANT MAINT materials,spares 1 PLANT MAINT reviews j updates PMs&WO PLANT MAINT: I__ PLANT MAINT reviews& ifecycle status c 1 reliability I construction submittals As needed,PLANT Templates in reviews training 1 returns comments nameplate info e I electrical and instrumentation I for: g I p Cityworks MAINT sends a I 1 outline&agenda 1 ++ 1 I Maintenance interim asset I 1 r- participates in training 1 1 I equirements registry changes to y I 1 (PM's) GIS updates PMs&WO IY I I templates in Cityworks 1 I Spares 1 Nameplate data 1 I I in 1 I 1 E I I 1 1 I OPS reviews design submittals: I OPS reviews OPS: 1 OPS reviews&returns c W T j Controls concepts j construction reviews training outline 1 comments CA comments Field walk I submittals: &agenda CPS reviews the final 1 Interface with ex facilities and r Integration with participates in training 0&M Manual and Q L— egiup,shutdowns ex systems updates SOPS,round updates SOPs,round • aC U Layouts shutdowns sheets,etc. sheets and other c Operability operator info aa aai Construction constraints and a 3 � sequence W AM/GIS staff assigns CA 0 AM/GIS staff generates asset/position numbers/ asset list in Process/ names,review asset AM/GIS staffupdates the GIS N I /GIS staff updates the GIS asset registry U Sub Process/System registry, submittals and requested by Eng. provides feedback.Staff (modified/overhauled assets) asset registry C7 6 Sends to Eng. updates the GIS asset a registry (new/reserved assets) 174 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 410 of 461 Page 177 of 195 10. ASSET MANAGEMENT PROCEDURES MANUAL These are located on Laserfiche under SOP/Engineering/Planning and Development: # Title Last Author Updated PDS-AM-01 Cityworks Custom Business Process Manual 4/7/2017 Dana Lawson PDS-AM-02 PDS-AM-02 Employee Set-Up and 2/16/2017 Dana Lawson Modification PDS-AM-03 PDS-AM-03 Updating Labor and Overhead 2/16/2017 Dana Lawson Rates Annually PDS-AM-04 Holiday Set-up and Modification 2/16/2017 Dana Lawson PDS-AM-05 Report Set-up and Modification 4/7/2017 Dana Lawson PDS-AM-06 Preventive Maintenance Set-up and 2/16/2017 Dana Lawson Modification PDS-AM-07 Crew Set-Up and Modification (Domain 2/16/2017 Dana Lawson Viewable) PDS-AM-08 Attachment Set-Up and Modification for 2/16/2017 Dana Lawson Templates PDS-AM-09 Project Set-Up and Modification 2/16/2017 Dana Lawson PDS-AM-10 Employee Skill Set-Up and Modification 4/7/2017 Dana Lawson PDS-AM-11 Equipment Set-Up and Modification 2/16/2017 Dana Lawson PDS-AM-12 Contractor Set-Up and Modification 2/16/2017 Dana Lawson PDS-AM-13 Material Set-Up and Modification 2/16/2017 Dana Lawson PDS-AM-14 Initiating 2/16/2017 Dana Lawson PDS-AM-15 Emergency (bypasses normal procedures) 2/16/2017 Dana Lawson PDS-AM-16 Screening 2/16/2017 Dana Lawson PD M-17 Planning 2/16/2017 Dana Lawson PDS-AM-18 Scheduling 1/19/2017 Dana Lawson PDS-AM-19 Assigning 2/16/2017 Dana Lawson PDS-AM-20 Re-Assigning Work 2/16/2017 Dana Lawson M-21 QA, Reviewing, and Closing 2/16/20217 Dana Lawson PDS-AM-22 Inspecting the Vehicle or Equipment 2/16/20217 Dana Lawson PDS-AM-23 Performing 2/16/20217 Dana Lawson PDS-AM-24 Completing QA Returned 2/16/20217 Dana Lawson PDS-AM-25 CMMS Section 2/16/20217 Dana Lawson 175 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 411 of 461 Page 178 of 195 PDS-AM-26 Paving Work Orders 2/16/20217 Dana Lawson PDS-AM-27 Overflow Response 2/16/20217 Dana Lawson PDS-AM-28 Fleet Services 2/16/20217 Dana Lawson PDS-AM-29 Plant Operations Division 2/16/20217 Dana Lawson PDS-AM-30 Process Control-Control Systems Section Draft 2/16/20217 Dana Lawson PDS-AM-31 Safety Section Draft 2/16/20217 Dana Lawson PDS-AM-32 Request for Buildings & Groups Shop 2/16/20217 Dana Lawson PDS-AM-33 Request for CCTV Section 2/16/20217 Dana Lawson PDS-AM-34 Risk Management Dra 2/16/20217 Dana Lawson PDS-AM-35 Updating GIS Map in CCTV Trucks 2/16/20217 Dana Lawson PDS-AM-36 Cityworks - TP Maintenance Rotating Assets 9/7/2018 Ian Morales PDS-AM-37 Loading Collection System and Recycled 5/22/2020 Ian Morales Water Distribution Asset Documents to L Laserfiche PDS-AM7-38 Loading Treatment Asset Documents to 6/3/2020 Ian Morales Laserfiche PDS-AM-39 Process Developer Job 12/29/2020 Mike Matthews PDS-AM-40 Parcel Editing —Merge Parcels 12/29/2020 Mike Matthews PDS-AM-41 Parcel Editing —Split Parcel 12/29/2020 Mike Matthews PDS-AM-42 Parcel Editing - Boundary Line Adjustment 12/29/2020 Mike Matthews PDS-AM-43 Parcel Editing -Convert Segmented Lines to 12/29/2020 Mike Matthews Curve PDS-AM-44 Parcel Editing -Convert Segmented Lines to 12/29/2020 Mike Matthews Curve (Adjacent to Right-of-Way) PDS-AM-45 Downloading Record Drawings from 12/28/2020 Khae Bohan Laserfiche PDS-AM-46 Creating and Editing Vicinity Maps from a 1 1/11/2020 Mireya Martin Template in ArcGIS Pro MNT-RE-1001 Asset Management and Maintenance 12/5/2022 Neil Meyer/Zach Lee Definitions MNT-RE-1002 Best Practices Risk Reduction 1/3/2023 Neil Meyer/Zach Lee MNT-RE-1003 Proactive Maintenance Loop 1/3/2023 Neil Meyer/Zach Lee MNT-RE-1004 Personal Employment Cycle 1/3/2023 Neil Meyer/Zach Lee MNT-RE-1005 FRACAS 1/3/2023 Neil Meyer/Zach Lee 176 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 412 of 461 Page 179 of 195 1 1 . BUSINESS CASE EVALUATION Central San's Engineering Department developed a suitable CIP validation framework and methodology. Business Case Evaluation (BCE) considers all aspects of asset ownership, including levels of service, capital costs, and O&M costs and incorporates non-cost factors. The figure below shows the process flow diagram for the BCE procedure employed at Central San. PROPOSED STANDARD BCE PROCESS FLOW efine DbGoals FunctionalPreparesStep 4c- San + i �GIB Index ' Recommendations ' Peer &NeDd Steps Assessment Review Review Step r Determinei ----- Key BCE Elements: 1. Define Problem:Goals and Project Drivers) 2. Functional Requirements &Condition Assessment 3. Determine Potential Alternatives 4.Analysis of Potential Alternatives a. Economic j b. Non-economic c. Summary of Cost-Benefit(C/B) Index Analysis 5.Pilot Test as Applicable 5.Recommendations Project Proceeds to PreliminaryDesiyn a. Present Recommendations b. Proceed to Design U 177 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 413 of 461 Page 180 of 195 • Map: Prepare a large map, if possible. Provide concept plan or sketch.] TECHNICAL ANALYSIS [Outline and evaluate options] Options [Consider:What are the options for meeting the project objectives?Are there creative or unconventional options to consider?What is the outcome of doing nothing?] Table 1 below lists the initial set of options discussed with the <e.g. Plant Maintenance Division, Collection System Operations Division, Capital Projects Division, etc. . Table 1.Initial Options No. Option Description A Do nothing continue existing O&M B Operate Differently C Maintain Differently D Repair/Rehab E I Replace F I Decommission Technical Factors [Consider the pros and cons of each option:What are the potential uncertainties?What are the risks to success?Any institutional barriers that exclude certain options? Consider the environmental impacts and permitting issues: What are the environmental impacts?What permits may be required?] Technical factors limiting the selection of feasible options include: <e.g. physical constraints, pavement moratorium, weather conditions, environmental or social impacts, etc.> Other agencies issues that limited the selection of feasible options included: <regulatory requirements, permit restrictions. [Are other agencies involved? If so, list them and what their issues are.] Further review and discussion indicated that option W above was not feasible due to... and option Y was eliminated due to Subsequently, Table 2 summarizes the feasible options that remained for further analysis. Table 2.Short-listed Options No. Option Description Pros Cons Maintain Differently Repair/Rehab Replace N:\EnvrSRV\AM-GIS\8238-AM Prog Dev\26 Reports\1 District Reports\Templates\Drafts\Business Case Evaluation for CIP.docx 178 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 414 of 461 Page 181 of 195 RISK ANALYSIS The primary failure mode for this problem/asset/project is: <pick one: physical mortality, regulatory or safety compliance, capacity, level of service, efficiency > The consequence of failure for this problem/asset/project is<describe the consequence(s) based on the primary failure mode above, e.g. service interruption, overflow, increased energy consumption, road closure, regulatory/safety violation, unsafe conditions, etc.>. The likelihood of failure for problem/asset/project is <e.g. XX% probability of failure of the XXX within X years>. [Discuss the relative risk reduction of the various options] Option X maximizes the risk reduction to the District. ECONOMIC ANALYSIS [Show capital and O&M costs for options. Present data in summary table. Consider the following: • Long-term O&M and other ongoing costs • Multi-year costs o Use Present Value at a range of discount rates (low, best, and high) o Include total capital costs o Include long-term O&M o Include any other lifecycle costs o Include indirect and intangible community costs • Benefits o Quantify if possible: direct, indirect and monetary o Highlight intangible (difficult to quantify) benefits • Net Present Value (NPV) o Apply to all costs and benefits that are quantifiable in dollar values o Alternatively, show project cost-effectiveness when benefits cannot be monetized] The short-listed options from Table 2 were evaluated using <pick one: present value (discounted), net present value (discounted), lifecycle cost, or triple bottom line> as shown in Table 3. Table 3.Life Cycle Cost Projections for Feasible Options Rates Discount Rate MARR' Total Ave Annual $ Option X.Description N:\EnvrSR\AAM-GIS\8238-AM Prog Dev\26 Reports\1 District Reports\Templates\Drafts\Business Case Evaluation for CIP.docx 179 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 415 of 461 Page 182 of 195 Duration of option(yrs) Capital Operations Maintenance Total Costs PV Total Costs Annuity of PV2 Option Y.Description Duration of option(yrs) Capital Operations Maintenance Total Costs PV Total Costs Annuity of PV2 Option Z. Description Duration of option(yrs) Capital Operations Maintenance Total Costs PV Total Costs Annuity of PV2 Notes Minimum Acceptable Rate of Return(consider current investment rate for investment of equivalent duration) 2 Annuity of PV:The series of uniform payments that will recover an initial investment(principal). Option X has the highest net present value (indicating it generates the highest net value to the community). Option Y has the lowest lifecycle cost,where the present value of costs is annualized to appropriately compare options of different durations. The decision rule here is to accept the recommendation with the lowest lifecycle cost, which in this case is option Y, followed by option Z. RECOMMENDATION [Describe your recommendation. Which option is recommended and why?What is the proposed budget and schedule?Where will the funding come from?] Option <X> is the recommended solution because <it has the highest combined ranking from the risk, economic, and technical analyses; and there is a high probability of near term of failure and its associated high consequence due to....> <e.g. project is linked to another project, emergency/urgent situation...> N:\EnvrSRV\AM-GIS\8238-AM Prog Dev\26 Reports\1 District Reports\Templates\Drafts\Business Case Evaluation for CIP.docx 180 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 416 of 461 Page 183 of 195 Option Analysis Rankings Total Risk Economic Technical C 1 1 1 3 D 2 3 3 8 E 3 2 2 This project has been reviewed using the Project Prioritization Form (see Attachment A) and is categorized as <Critical, Very High, High, Medium> priority. I recommend including this project in the Capital Improvement Plan for FY 20XX-XX under the XXX Program. The estimated capital cost in current (non-inflated) dollars is $X million, <describe funding source or requiring an annual debt service payment of$X for X years at X%>. The estimated annual O&M cost is$X. The anticipated project schedule is: Task Start Duration Preliminary Design MM-YYYY #months Design MM-YYYY #months Permits MM-YYYY #months Bid MM-YYYY #months Construction MM-YYYY #months N:\EnvrSR\AAM-GIS\8238-AM Prog Dev\26 Reports\1 District Reports\Templates\Drafts\Business Case Evaluation for CIP.docx 181 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 417 of 461 Page 184 of 195 12. PUMPING STATIONS CONSEQUENCE OF FAILURE MATRIX Value Service Requirements Reliability Contingency Requirements Options 1 <1 Unlikely to reach Bypass pumping 2 1-2 waters of state and capability 3 2-3 not in drinking watershed 4 3-4 Category 1 or East Bay Small enough to 5 4-5 Municipal Utilities vactor wet well 6 5-6 District 7 6-7 8 7-8 Category 1 + East Bay No contingency 9 8-9 Municipal Utilities options 10 9-10 District Pumping Station Service Reliability Contingency Requirements (50%) Requirements • . • Acacia 1 10 1 3 Bates Blvd 1 1 5 2 Buchanan North LS 1 1 1 1 Buchanan South LS 1 1 1 1 Clyde 1 5 1 2 Concord Industrial 1 5 1 2 Fairview 4 5 1 4 Flush Kleen 1 10 1 3 Lower Orinda 3 10 1 4 Maltby 4 5 1 4 Martinez 4 5 1 4 Moraga 5 10 1 5 Orinda Crossroads 5 10 1 5 San Ramon 10 5 1 7 Sleepy Hollow 1 5 1 2 Via Roble 1 1 5 2 Wagner Ranch (private) 1 1 5 2 Wilder Lower(private) 1 1 5 2 Wilder Upper(private) 1 1 5 2 182 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 418 of 461 Page 185 of 195 13. CONDITION ASSESSMENT RESULTS AND SERVICE LIFE Information on Condition Assessment Results for sewer assets is in the CWMP (Carollo Engineers, 2017). Information on the Treatment Plant assets for the Steam System is under the Steam Condition Assessment Report—Steam and Aeration Blower Renovations Project. 183 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 419 of 461 Page 186 of 195 14. ENTERPRISE RISK INDEX Enterprise Risk Management (ERM) refers to an organization's ability to understand, express, and control the level of risk taken to meet its strategic goals and establishes accountability for those risks. Central San's program began several years ago when managers and staff developed operational risk inventories for their programs, services, and functions. These risks were integrated into the Executive Team's strategic risk inventory. The ERM team periodically reviews and updates this inventory, monitors the effectiveness of mitigations, and considers new mitigation techniques to manage Central San's evolving operational and strategic risks. ERM includes organizations' processes to manage risks and advance opportunities to achieve their objectives, typically identified in a strategic plan. ERM provides a framework for risk management which involves identifying risks and opportunities, assessing them in terms of likelihood and magnitude of impact, determining a response strategy, and establishing a monitoring process. ERM has evolved to address the needs of various stakeholders, including boards, who want to understand the broad spectrum of risks facing complex organizations to ensure they are appropriately managed. Regulators and debt rating agencies have also increased their scrutiny of companies' risk management processes, with Standard & Poor's introducing ERM compliance into their rating criteria in 2009. Organizations have traditionally managed risks in a distributed way, with various functions identifying and managing risks. Before ERM, these efforts were not typically centrally coordinated or reported. A central goal of ERM is improving this capability and coordination while providing summary-level reporting to provide a unified picture of risk for stakeholders and improving the organization's ability to manage these risks effectively. Establishing an ERM program typically involves the following: • Identifying executive sponsors for ERM • Establishing a common risk language or glossary • Describing the entity's risk appetite (i.e., risks it will and will not take) • Identifying and describing the risks in a " risk inventory" • Implementing a risk-ranking methodology to prioritize risks within and across functions • Establishing a risk committee and/or responsible risk officer to coordinate certain activities of the risk 15. DEVELOPMENT OF THE ENTERPRISE RISK INVENTORY In January 2020, staff presented Central San's initial Strategic Risk Inventory to the Board. At that time, it was noted that the risk inventory would be used as the foundation for an Enterprise Risk Management (ERM) Program. Central San Strategic uses its Risk Inventory for two purposes: 1. As an input to the Internal Auditor' s annual work plan. 2. For monitoring, control, and reporting on risks. The ERM Team meets twice annually to discuss progress on mitigating the risks identified in the Strategic Risk Inventory. Updates to the Strategic Risk Inventory are regularly reported to the Administration Committee and Board. 184 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 420 of 461 Page 187 of 195 Risk Probability Severity Mitigation Speed of Total = Risk Rank D- D. Onset Score Economic7 8 2 8 ASSET 1 Uncertainty Global 6 ASSET 3 5 ASSET 2 Pandernic Internal 4 3 2 7 ASSET 3 Controls Failure Each risk has been assigned a composite score of up to 40 points consisting of four elements, each with up to 10 points: frequency, impact, remaining work needed to mitigate the risk, and speed of onset. Each risk has been assigned to a member of senior management who is responsible for developing a mitigation plan for each risk. The mitigation plan describes elements of the risk and specifies actions to be taken to reduce, control, or eliminate the risk. Finally, to integrate the two risk registers (operational and strategic), each risk on the original operational risk inventory was also associated with a strategic risk for tracking and continued efforts to address those risks. Current Strategic Risks (28): Risk Owner Description Operations Service or Product Failure Slow Response to Customer Poor Jurisdictional Coordination Loss of Utilities/Supply Chain Continuity Threat/Pandemic Loss of Major Asset Physical Security Breach 185 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 421 of 461 Page 188 of 195 Risk Owner Description Engineering Environmental Risk New/Proposed Regulations/Legislation Natural Disaster Major Spill Poor Coordination on Large Projects Risk Owner Description No Finance & Admin Poor Customer Communications Self-insurance/Reserve Insufficiency Failure of Internal Controls Economic Downturn/Recession Need for Large Rate Increase Loss of Major Customer/Partner Higher Borrowing Costs/Loss of Tax-Exempt Bond Status Social/Political Risk (Civil Unrest, etc.) Cyber Security Large Tech Implementation Failure External Data Connectivity Risk Failure to Adopt New Technology Risk Owner Description Foss of Life/Major Injury Work Stoppage Changing Workforce Change Readiness Risk 186 Central San Asset Management Plan December 21, 2023 Regular Board Meeting Agenda Packet- Page 422 of 461 Page 189 of 195 ATTACHMENT 2 December 21,2023 Asset Management Plan Update Khae Bohan,Asset Management Program Administrator Nitin Goet,Operations Optimization i Division Manager - Objectives 1Pthe rovide Asset Plan ASSET ^° �• a Board MANAGEMENT PLAN 2update for Central A2023 Asset Management � v Plan FF ,� : �., A Ak--- n led, December 21, 2023 Regular Board Meeting Agenda Packet- Page 423 of 461 1 Page 190 of 195 Purpose of the Asset Management Plan Provide an overview of Central current condition of assets and their overall performance; Document system operation and maintenance strategies and documented pro ---- Describe levelstandards, performance and adopted soma oo 00 Proposed actions to implement the aboveoperation,maintenance,and strategies; . �� ray.'•�o� +asoa���a�rya .'r'""w c��'•"°wo"• 1"°"".mo�,a�`maw Describe - • .- engagement . communication plan. h L wa . ••n'w'"°'""'"`.,y spa sta Quick Glance at the Asset Management Plan Risks Decision Making Criteria:Drivers and Life • Risk Definition Cycle Management Standards and Level of Service Identifying the Risks Drivers • Board Policy Quantifying the Risk Aging Infrastructure • EPA's(Environmental Protection Condition Assessment Program Regulatory Agency)10-Step Process for Executive Level Risk Indices Sustainability Developing an Asset Management Optimization Plan / Capital Improvement Program • ISO(International Organization for Asset Lifecycle Standardization)55000 series ASSET Remaining Useful Life • Compliance and Regulatory Data Management and Utilization for Requirements MANAGEMENT PLAN Decision-Making • LeveL of Service:Metrics from Strategic PLan ���� e< ... Assets Vre.Po, anagement Information • Asset Definition and Tools • Sewer Gravity Main tat-Dynamic Web-Based Map Staff Development • Sewer Force Main and Data Access Asset Management Approaches Personnel Advancement • Pumping Stations Single Asset Registry General Approaches Educational Resources • Treatment Plant Equipment Asset Hierarchies • Operations Internship Program and Piping CMMS(Computerized Maintenance • Maintenance Mentorship Program • Recycled Water Distribution Management System) Specific Approaches Externship Program • General Improvements CCTV(Closed-Circuit Television) Collection System Management and Supervisory Inspection System Automatic Vehicle Treatment Plant Academy Locating Recycled Water Career Development Program • USA(Underground Service Alert)Locate SOP(Standard Operating Procedures) Performance Recognition, Tickets Monitoring and Managing System Breakdowns Appreciation,and Beyond • Spare Parts Management Application Updating and Maintaining Procedures Conferences and Training • Reliability Engineering Report Performance Monitoring and Reporting Team Accomplishments • Document Management System Operational Resilience and Disaster Preparedness • Risk and Flow Modeling Software December 21, 2023 Regular Board Meeting Agenda Packet- Page 424 of 461 2 Page 191 of 195 Asset Management & Reliability Engineering Timeline Focus on Systems and Best Asset Note Abbreviations: Technology 0 AM AM(Asset Management) rogramAMIP(Asset Management -- Implementation Plan) Asset - MRO(Maintenanceand , Decision Repair,and Operations) Asset Condition Management and Risk (2016-2017) Management Best Work Execution Continuous Preventive Maintenance Improvement and Program Management Planning and Scheduling Operator Driven [rtyworks Reliability Spares Engineering for Defect Elimination Best Reliability ManagementMaintenance Program Before 2016 Sussex Field Transfer System Using CMMS software no longer offering c..l support .Multiple asset registries K Workflow consisted of paper map and manual digital entries §�;,,; — After Cityworks"go-live"September 2016(office) __ iPads for field crews available December 2016 Treatment Plant"go-live" in January 2017 - Historical (legacy)data imported (10 years) --_:.-- Digital process — _-- = — Better scheduling workflow More efficient in addressing work orders in r t field using mobile devicesI�l December 21, 2023 Regular Board Meeting Agenda Packet- Page 425 of 461 3 Page 192 of 195 Following These Standards EPA's 10-Step Process for Developing an Asset Management Plan ISO 55000 Series Effective Utility Management Uptime Elements Ali ' $P ° 5--PAM O Ri Ak AM Om A Ci P ISO VP Asset Management Information Systems and Tools b,. Works ICM InfoMastw Cltyworks DYNAC tikY�Yl er Na,men•Me �- Pia.i Can[rN �.. Nyarwia Cap.ln• ♦ y„• woe.i R\\ wwMM1w OrinlrM eesd ORACLE °ate r�ra rr srwee W ............ KTY M.R.aler/ y�rrr Ryry �VB([iG15 Studio ••••••••••••• �,,,,,,,«,� ® ..... i Tool WA hw&Aal YQN 4� Rm lbw — WWfrypMiik! .. IAp$bMCN 8 December 21, 2023 Regular Board Meeting Agenda Packet- Page 426 of 461 4 Page 193 of 195 PrdaCdve MainsenanCe CdnSnuous ImprDvement Loop Asset Management Strategies F�� wok 11RACAs-Ra'�wre rwn,irw�aon Reporting Analysis Reliability Centered Maintenance Program z_sys,e'm ssep s: lllll,i,+++r..... swn z: Work order Maim...nce close Ou! Collection System Operations (CSO) Preventative Maintenance Program soep a; stop s: Work Ma,nlona Inspections via CCTV Quality Assurance/Control Program Rehabilitation and Replacement Program Risk-Based Planning Treatment Plant Reliability Strategy and Reliability Engineering Asset Condition Information and Program Work Execution (Preventative Maintenance Programs) Workforce Development Personnel Employment Cycle Step 7: Hew nuns are reuuired to Cemmr PesomelarenalncatoP.r : Promotlonand Son Andaolwbwrdeddy , nmmoted and re mr'o e'rak''„`s Retirement H�nen Resauce and r r,+y,riC-Rveraoe wma ro RA C Wn"twdpn FA-M 25%ir-ml MHnc-k imerns in each !k 1 prrsriul'ons veer rwin:allertleA!% 1 Step 6: Step 1: a. edn oaaro.�a Succession Planning Recruitment and Onboarding I` rraln rper-1rq I:pded rn perl w prepare Them tar s+"M50rY roles 01 Metric-EnemveusemCen ml Strategic Goal 1 ,arllk ,Awneoer sceawISand cross _�1/_ rnnmr. %crvey�IMM vwlnm r ss swr anendwnce In Workforce Development: malning and dn.�rJnpma t ..4—,. Recruit,empower,and `"e rI'" progmm_anp. B�rsnp prtxgam Step g: engage a highlytrained Step Career Advancement Shop Training and and safe workforce Development Checklist 1 F"swwI rc[ervr annual S91c[r and - lecMcal tiaininp[v kvin xrd rr_riim cc�k I�; �� Mcmt :n.CBtt(:mined Relatillly LaMr Step 4; C+:ion Cry'or MaIfACnanCC A'HCIIa[s lty Step 3: Annual Training wrmezr,onar.rcrineo: sraa l:.v,orn,a 9 Personikr iercve ou vucwve .- ymr Ennwirneir A,slxinrcricerLned. Activities Annual Performance ree,wackrmarsn�r.,�rr rya - �O_Amncond,oW,rnW 0,inu daca+p ��Appraisals and Career Mmrc %NcnamameApprx� b ran'wreminrm�x,�Wrar0— Development Review comykedarb.-I81Ods IL* Comr um pamaparon December 21, 2023 Regular Board Meeting Agenda Packet- Page 427 of 461 5 Page 194 of 195 Benefits of Robust Asset Management Program CSO Enhanced efficiency and visibility TOTAL SPILLS BY YEAR(ALL CATEGORIES) Documented strategies, processes, and procedures Improved lifecycle asset management Rehabilitate assets to extend lifespan of facilities and equipment Enhanced data-driven decision making Better resource planning and forecasting Plant Maintenance Before After Examples of tangible benefits: r1/1/2022 Reactive 10° 3% CSO-total spills decreased overall Corrective 20% 29% Plant Maintenance- reactive work orders Non-Maintenance 59/o 2910 decreased last two years Predictive 6% 5% Preventative 58% 62% Blank .03% .05% Note:Percentage rounding Asset Management Plan Initiatives (2023-28) No. nt Pla Planned Duration ASSET Start (montbo MANAGEMENTROADMAP 1 CompLete the Asset Registry for Treatment Plant Piping 2023 16 s: Underground Utility .9e 2021 2 Continue Development of Data Analytics 2023 12 0 3 Continue Comprehensive Asset Evaluation for the 2022 Ongoing 2023 Prioritizations of Vertical Assets 4 Digitize Operator Round Sheets 2024 12 to 16 2023 approximately 5 Smart Utility initiatives 2025 36 2024 Complete standardization of Smart Motor Control Centers across Central San • Pilot Asset Health/Condition Monitoring TechnoLogy • Asset Handoff Tracking and Monitoring 2026 • AppLy/Pilot Artificial Intelligence/Machine Learning 2027 Where AppLicabLe 6 Dynamic Asset Management Plan 2028 12p 2028 2023 �J 12 2028 December 21, 2023 Regular Board Meeting Agenda Packet- Page 428 of 461 6 Page 195 of 195 Acknowledgement `^ 4 This has been a collaborative effort with various staff's input and feedback. We thank everyone who contributed. Questions? *-POO- December 21, 2023 Regular Board Meeting Agenda Packet- Page 429 of 461 7